Search Results for author:
Found 19 papers, 8 papers with code
Toward a Diffusion-Based Generalist for Dense Vision Tasks
In this paper, we explore diffusion-based vision generalists, where we unify different types of dense prediction tasks as conditional image generation and re-purpose pre-trained diffusion models for it.
How Good is my Video LMM? Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMs
Recent advancements in Large Language Models (LLMs) have led to the development of Video Large Multi-modal Models (Video-LMMs) that can handle a wide range of video understanding tasks.
GiT: Towards Generalist Vision Transformer through Universal Language Interface
Due to its simple design, this paradigm holds promise for narrowing the architectural gap between vision and language.
Ranked #2 on
Video Captioning
on MSVD-CTN
(using extra training data)
FocusCLIP: Multimodal Subject-Level Guidance for Zero-Shot Transfer in Human-Centric Tasks
We propose FocusCLIP, integrating subject-level guidance--a specialized mechanism for target-specific supervision--into the CLIP framework for improved zero-shot transfer on human-centric tasks.
Ranked #1 on
Age Classification
on EMOTIC
Learning to Prompt with Text Only Supervision for Vision-Language Models
While effective, most of these works require labeled data which is not practical, and often struggle to generalize towards new datasets due to over-fitting on the source data.
SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance
In SemiVL, we propose to integrate rich priors from VLM pre-training into semi-supervised semantic segmentation to learn better semantic decision boundaries.
Ranked #1 on
Semi-Supervised Semantic Segmentation
on PASCAL VOC 2012 732 labeled
(using extra training data)
SILC: Improving Vision Language Pretraining with Self-Distillation
However, the contrastive objective used by these models only focuses on image-text alignment and does not incentivise image feature learning for dense prediction tasks.
Introducing Language Guidance in Prompt-based Continual Learning
While the model faces a disjoint set of classes in each task in this setting, we argue that these classes can be encoded to the same embedding space of a pre-trained language encoder.
I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification
Our proposed model, I2MVFormer, learns multi-view semantic embeddings for zero-shot image classification with these class views.
Learning Attention Propagation for Compositional Zero-Shot Learning
CAPE learns to identify this structure and propagates knowledge between them to learn class embedding for all seen and unseen compositions.
I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification
In order to distill discriminative visual words from noisy documents, we introduce a new cross-modal attention module that learns fine-grained interactions between image patches and document words.
3D Compositional Zero-shot Learning with DeCompositional Consensus
Parts represent a basic unit of geometric and semantic similarity across different objects.
Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions.
Learning Graph Embeddings for Compositional Zero-shot Learning
In compositional zero-shot learning, the goal is to recognize unseen compositions (e. g. old dog) of observed visual primitives states (e. g. old, cute) and objects (e. g. car, dog) in the training set.
Open World Compositional Zero-Shot Learning
After estimating the feasibility score of each composition, we use these scores to either directly mask the output space or as a margin for the cosine similarity between visual features and compositional embeddings during training.
Reliable Fidelity and Diversity Metrics for Generative Models
In this paper, we show that even the latest version of the precision and recall metrics are not reliable yet.
Deep Learning Under the Microscope: Improving the Interpretability of Medical Imaging Neural Networks
In this paper, we propose a novel interpretation method tailored to histological Whole Slide Image (WSI) processing.
Data Augmentation with Manifold Exploring Geometric Transformations for Increased Performance and Robustness
Compared with traditional augmentation methods, and with images synthesized by Generative Adversarial Networks our method not only achieves state-of-the-art performance but also significantly improves the network's robustness.
A Multi-faceted OCR Framework for Artificial Urdu News Ticker Text Recognition
We present the first comprehensive data set, to our knowledge, for Urdu news ticker recognition, collected from 41 different news channels.
