Search Results for author:
Found 20 papers, 13 papers with code
Pixel Motion as Universal Representation for Robot Control
We present LangToMo, a vision-language-action framework structured as a dual-system architecture that uses pixel motion forecasts as intermediate representations.
Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation
We present Seg-TTO, a novel framework for zero-shot, open-vocabulary semantic segmentation (OVSS), designed to excel in specialized domain tasks.
Open Vocabulary Semantic Segmentation
Open-Vocabulary Semantic Segmentation
+1
LatentCRF: Continuous CRF for Efficient Latent Diffusion
Latent Diffusion Models (LDMs) produce high-quality, photo-realistic images, however, the latency incurred by multiple costly inference iterations can restrict their applicability.
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy
In this work, we introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations and enables an efficient transfer of a pretrained VLM into a powerful VLA, motivated by the success of visual instruction tuning in Computer Vision.
Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA
Long-form videos that span across wide temporal intervals are highly information redundant and contain multiple distinct events or entities that are often loosely related.
Ranked #2 on
Zero-Shot Video Question Answer
on IntentQA
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs
Integration of Large Language Models (LLMs) into visual domain tasks, resulting in visual-LLMs (V-LLMs), has enabled exceptional performance in vision-language tasks, particularly for visual question answering (VQA).
Ranked #1 on
Zero-Shot Region Description
on RefCOCOg-test
Understanding Long Videos with Multimodal Language Models
Our resulting Multimodal Video Understanding (MVU) framework demonstrates state-of-the-art performance across multiple video understanding benchmarks.
Language Repository for Long Video Understanding
In this paper, we introduce a Language Repository (LangRepo) for LLMs, that maintains concise and structured information as an interpretable (i. e., all-textual) representation.
Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning
Self-supervised representation learning has been highly promising for histopathology image analysis with numerous approaches leveraging their patient-slide-patch hierarchy to learn better representations.
Diffusion Illusions: Hiding Images in Plain Sight
We explore the problem of computationally generating special `prime' images that produce optical illusions when physically arranged and viewed in a certain way.
Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentors
In this work, we explore how an off-the-shelf text-to-image diffusion model, trained without exposure to localization information, can ground various semantic phrases without segmentation-specific re-training.
Perceptual Grouping in Contrastive Vision-Language Models
In this work we examine how well vision-language models are able to understand where objects reside within an image and group together visually related parts of the imagery.
Self-supervised Video Transformer
To the best of our knowledge, the proposed approach is the first to alleviate the dependency on negative samples or dedicated memory banks in Self-supervised Video Transformer (SVT).
On Improving Adversarial Transferability of Vision Transformers
(ii) Token Refinement: We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT.
Intriguing Properties of Vision Transformers
We show and analyze the following intriguing properties of ViT: (a) Transformers are highly robust to severe occlusions, perturbations and domain shifts, e. g., retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content.
Orthogonal Projection Loss
The CE loss encourages features of a class to have a higher projection score on the true class-vector compared to the negative classes.
Conditional Generative Modeling via Learning the Latent Space
Although deep learning has achieved appealing results on several machine learning tasks, most of the models are deterministic at inference, limiting their application to single-modal settings.
Extending Multi-Object Tracking systems to better exploit appearance and 3D information
Tracking multiple objects in real time is essential for a variety of real-world applications, with self-driving industry being at the foremost.
Bipartite Conditional Random Fields for Panoptic Segmentation
To tackle this problem, we propose a CRF model, named Bipartite CRF or BCRF, with two types of random variables for semantic and instance labels.
Combined Static and Motion Features for Deep-Networks Based Activity Recognition in Videos
We propose three schemas for combining static and motion components: based on a variance ratio, principal components, and Cholesky decomposition.
