Search Results for author:
Found 35 papers, 19 papers with code
Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos
Understanding fine-grained temporal dynamics is crucial in egocentric videos, where continuous streams capture frequent, close-up interactions with objects.
UIP2P: Unsupervised Instruction-based Image Editing via Cycle Edit Consistency
We propose an unsupervised model for instruction-based image editing that eliminates the need for ground-truth edited images during training.
Active Data Curation Effectively Distills Large-Scale Multimodal Models
Knowledge distillation (KD) is the de facto standard for compressing large-scale models into smaller ones.
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
By treating model parameters as tokens, we replace all the linear projections in Transformers with our token-parameter attention layer, where input tokens act as queries and model parameters as keys and values.
Toward a Diffusion-Based Generalist for Dense Vision Tasks
In this paper, we explore diffusion-based vision generalists, where we unify different types of dense prediction tasks as conditional image generation and re-purpose pre-trained diffusion models for it.
LocCa: Visual Pretraining with Location-aware Captioners
In this paper, we propose a simple visual pretraining method with location-aware captioners (LocCa).
Text-Conditioned Resampler For Long Form Video Understanding
In this paper we present a text-conditioned video resampler (TCR) module that uses a pre-trained and frozen visual encoder and large language model (LLM) to process long video sequences for a task.
Ranked #25 on
Video Question Answering
on NExT-QA
LIME: Localized Image Editing via Attention Regularization in Diffusion Models
A significant challenge within this domain is localized editing, where specific areas of an image are modified without affecting the rest of the content.
PALM: Predicting Actions through Language Models
Traditional methods heavily rely on representation learning that is trained on a large amount of video data.
SILC: Improving Vision Language Pretraining with Self-Distillation
However, the contrastive objective used by these models only focuses on image-text alignment and does not incentivise image feature learning for dense prediction tasks.
Detecting Adversarial Faces Using Only Real Face Self-Perturbations
Adversarial attacks aim to disturb the functionality of a target system by adding specific noise to the input samples, bringing potential threats to security and robustness when applied to facial recognition systems.
Learning Prototype Classifiers for Long-Tailed Recognition
In this work, we show that learning prototype classifiers addresses the biased softmax problem in LTR.
Ranked #8 on
Long-tail Learning
on CIFAR-100-LT (ρ=10)
Weakly-Supervised Domain Adaptive Semantic Segmentation With Prototypical Contrastive Learning
In this work, we propose a common framework to use different weak labels, e. g. image, point and coarse labels from target domain to reduce this performance gap.
Urban Scene Semantic Segmentation with Low-Cost Coarse Annotation
For best performance, today's semantic segmentation methods use large and carefully labeled datasets, requiring expensive annotation budgets.
CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution
We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features.
I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification
Our proposed model, I2MVFormer, learns multi-view semantic embeddings for zero-shot image classification with these class views.
I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification
In order to distill discriminative visual words from noisy documents, we introduce a new cross-modal attention module that learns fine-grained interactions between image patches and document words.
Attribute Prototype Network for Any-Shot Learning
While a visual-semantic embedding layer learns global features, local features are learned through an attribute prototype network that simultaneously regresses and decorrelates attributes from intermediate features.
Ranked #5 on
GZSL Video Classification
on ActivityNet-GZSL(main)
VGSE: Visually-Grounded Semantic Embeddings for Zero-Shot Learning
Our model visually divides a set of images from seen classes into clusters of local image regions according to their visual similarity, and further imposes their class discrimination and semantic relatedness.
3D Compositional Zero-shot Learning with DeCompositional Consensus
Parts represent a basic unit of geometric and semantic similarity across different objects.
Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions.
Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
Having access to multi-modal cues (e. g. vision and audio) empowers some cognitive tasks to be done faster compared to learning from a single modality.
A Closer Look at Self-training for Zero-Label Semantic Segmentation
Being able to segment unseen classes not observed during training is an important technical challenge in deep learning, because of its potential to reduce the expensive annotation required for semantic segmentation.
Ranked #8 on
Zero-Shot Semantic Segmentation
on PASCAL VOC
Learning Graph Embeddings for Compositional Zero-shot Learning
In compositional zero-shot learning, the goal is to recognize unseen compositions (e. g. old dog) of observed visual primitives states (e. g. old, cute) and objects (e. g. car, dog) in the training set.
Open World Compositional Zero-Shot Learning
After estimating the feasibility score of each composition, we use these scores to either directly mask the output space or as a margin for the cosine similarity between visual features and compositional embeddings during training.
Prototype-based Incremental Few-Shot Semantic Segmentation
Semantic segmentation models have two fundamental weaknesses: i) they require large training sets with costly pixel-level annotations, and ii) they have a static output space, constrained to the classes of the training set.
Attribute Prototype Network for Zero-Shot Learning
As an additional benefit, our model points to the visual evidence of the attributes in an image, e. g. for the CUB dataset, confirming the improved attribute localization ability of our image representation.
Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation
Few-shot learning aims to recognize novel classes from a few examples.
Analyzing the Dependency of ConvNets on Spatial Information
In this paper, we are pushing the envelope and aim to further investigate the reliance on spatial information.
Semantic Projection Network for Zero- and Few-Label Semantic Segmentation
In this paper we take this one step further and focus on the challenging task of zero- and few-shot learning of semantic segmentation.
Ranked #10 on
Zero-Shot Semantic Segmentation
on COCO-Stuff
f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
When labeled training data is scarce, a promising data augmentation approach is to generate visual features of unknown classes using their attributes.
Ranked #4 on
Generalized Zero-Shot Learning
on SUN Attribute
Feature Generating Networks for Zero-Shot Learning
Suffering from the extreme training data imbalance between seen and unseen classes, most of existing state-of-the-art approaches fail to achieve satisfactory results for the challenging generalized zero-shot learning task.
Ranked #6 on
Generalized Zero-Shot Learning
on SUN Attribute
Generalized Zero-Shot Learning
Generative Adversarial Network
Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly
Due to the importance of zero-shot learning, i. e. classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily.
Zero-Shot Learning -- The Good, the Bad and the Ugly
Due to the importance of zero-shot learning, the number of proposed approaches has increased steadily recently.
Latent Embeddings for Zero-shot Classification
We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image.
