Search Results for author:
Found 61 papers, 41 papers with code
On Large Multimodal Models as Open-World Image Classifiers
Despite this remarkable capability, most existing studies on LMM classification performance are surprisingly limited in scope, often assuming a closed-world setting with a predefined set of categories.
Training-Free Personalization via Retrieval and Reasoning on Fingerprints
We depart from existing work, and for the first time explore the training-free setting in the context of personalization.
Can Text-to-Video Generation help Video-Language Alignment?
Recent video-language alignment models are trained on sets of videos, each with an associated positive caption and a negative caption generated by large language models.
Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models
We demonstrate that visual embeddings of pre-trained VLMs exhibit a compositional arrangement, and evaluate the effectiveness of this property in the tasks of compositional classification and group robustness.
Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages
An old-school recipe for training a classifier is to (i) learn a good feature extractor and (ii) optimize a linear layer atop.
Group-robust Machine Unlearning
MIU minimizes the mutual information between model features and group information, achieving unlearning while reducing performance degradation in the dominant group of the forget set.
3D Part Segmentation via Geometric Aggregation of 2D Visual Features
Supervised 3D part segmentation models are tailored for a fixed set of objects and parts, limiting their transferability to open-set, real-world scenarios.
One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering
To mitigate this issue, we introduce a pseudo-rehearsal balancing module that aligns the generated data towards the ground-truth data distribution using either the question meta-statistics or an unsupervised clustering method.
Benchmarking XAI Explanations with Human-Aligned Evaluations
Our first key contribution is a human evaluation of XAI explanations on four diverse datasets (COCO, Pascal Parts, Cats Dogs Cars, and MonumAI) which constitutes the first large-scale benchmark dataset for XAI, with annotations at both the image and concept levels.
GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models
OpenBias detects and quantifies biases, while GradBias determines the contribution of individual prompt words on biases.
The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models
Vision-Language Models (VLMs) combine visual and textual understanding, rendering them well-suited for diverse tasks like generating image captions and answering visual questions across various domains.
One-Shot Unlearning of Personal Identities
To this extent, existing MU approaches assume complete or partial access to the training data, which can be limited over time due to privacy regulations.
Automatic benchmarking of large multimodal models via iterative experiment programming
Finally, the LLM refines the report, presenting the results to the user in natural language.
Frustratingly Easy Test-Time Adaptation of Vision-Language Models
Thanks to its simplicity and comparatively negligible computation, ZERO can serve as a strong baseline for future work in this field.
Less is more: Summarizing Patch Tokens for efficient Multi-Label Class-Incremental Learning
Since independent pathways in truly incremental scenarios will result in an explosion of computation due to the quadratically complex multi-head self-attention (MSA) operation in prompt tuning, we propose to reduce the original patch token embeddings into summarized tokens.
Vocabulary-free Image Classification and Semantic Segmentation
To address VIC, we propose Category Search from External Databases (CaSED), a training-free method that leverages a pre-trained vision-language model and an external database.
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set.
MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning
In this challenging setting, the transferable representations already encoded in the pretrained model are a key aspect to preserve.
Harnessing Large Language Models for Training-free Video Anomaly Detection
Video anomaly detection (VAD) aims to temporally locate abnormal events in a video.
Vision-by-Language for Training-Free Compositional Image Retrieval
Finally, we show that CIReVL makes CIR human-understandable by composing image and text in a modular fashion in the language domain, thereby making it intervenable, allowing to post-hoc re-align failure cases.
Ranked #1 on
Zero-Shot Composed Image Retrieval (ZS-CIR)
on GeneCIS
(A-R@1 metric)
PDiscoNet: Semantically consistent part discovery for fine-grained recognition
Fine-grained classification often requires recognizing specific object parts, such as beak shape and wing patterns for birds.
Iterative Superquadric Recomposition of 3D Objects from Multiple Views
We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views without training a model that uses 3D supervision.
Image-free Classifier Injection for Zero-Shot Classification
We achieve this with our proposed Image-free Classifier Injection with Semantics (ICIS) that injects classifiers for new, unseen classes into pre-trained classification models in a post-hoc fashion without relying on image data.
On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers
State-of-the-art rehearsal-free continual learning methods exploit the peculiarities of Vision Transformers to learn task-specific prompts, drastically reducing catastrophic forgetting.
ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models
We propose ProbVLM, a probabilistic adapter that estimates probability distributions for the embeddings of pre-trained VLMs via inter/intra-modal alignment in a post-hoc manner without needing large-scale datasets or computing.
Vocabulary-free Image Classification
We thus formalize a novel task, termed as Vocabulary-free Image Classification (VIC), where we aim to assign to an input image a class that resides in an unconstrained language-induced semantic space, without the prerequisite of a known vocabulary.
If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection
Despite their impressive capabilities, diffusion-based text-to-image (T2I) models can lack faithfulness to the text prompt, where generated images may not contain all the mentioned objects, attributes or relations.
Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval
Representation learning for sketch-based image retrieval has mostly been tackled by learning embeddings that discard modality-specific information.
Relational Proxies: Emergent Relationships as Fine-Grained Discriminators
Fine-grained categories that largely share the same set of parts cannot be discriminated based on part information alone, as they mostly differ in the way the local parts relate to the overall global structure of the object.
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey
Semi-supervised learning and unsupervised learning offer promising paradigms to learn from an abundance of unlabeled visual data.
Abstracting Sketches through Simple Primitives
Toward equipping machines with such capabilities, we propose the Primitive-based Sketch Abstraction task where the goal is to represent sketches using a fixed set of drawing primitives under the influence of a budget.
BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks
Moreover, many of the high-performing deep learning models that are already trained and deployed are non-Bayesian in nature and do not provide uncertainty estimates.
KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning
The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images, given only a subset of them during training and no prior on the unseen compositions.
Attention Consistency on Visual Corruptions for Single-Source Domain Generalization
Generalizing visual recognition models trained on a single distribution to unseen input distributions (i. e. domains) requires making them robust to superfluous correlations in the training set.
Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation
To tackle these issues, we introduce a novel incremental class learning approach for semantic segmentation taking into account a peculiar aspect of this task: since each training step provides annotation only for a subset of all possible classes, pixels of the background class exhibit a semantic shift.
Concurrent Discrimination and Alignment for Self-Supervised Feature Learning
Existing self-supervised learning methods learn representation by means of pretext tasks which are either (1) discriminating that explicitly specify which features should be separated or (2) aligning that precisely indicate which features should be closed together, but ignore the fact how to jointly and principally define which features to be repelled and which ones to be attracted.
On the Challenges of Open World Recognitionunder Shifting Visual Domains
Robotic visual systems operating in the wild must act in unconstrained scenarios, under different environmental conditions while facing a variety of semantic concepts, including unknown ones.
Detecting Anomalies in Semantic Segmentation with Prototypes
Current state of the art of anomaly segmentation uses generative models, exploiting their incapability to reconstruct patterns unseen during training.
Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions.
Cluster-driven Graph Federated Learning over Multiple Domains
Clustering may reduce heterogeneity by identifying the domains, but it deprives each cluster model of the data and supervision of others.
A Closer Look at Self-training for Zero-Label Semantic Segmentation
Being able to segment unseen classes not observed during training is an important technical challenge in deep learning, because of its potential to reduce the expensive annotation required for semantic segmentation.
Ranked #8 on
Zero-Shot Semantic Segmentation
on PASCAL VOC
Inferring Latent Domains for Unsupervised Deep Domain Adaptation
Most deep UDA approaches operate in a single-source, single-target scenario, i. e. they assume that the source and the target samples arise from a single distribution.
Boosting Binary Masks for Multi-Domain Learning through Affine Transformations
In this work, we provide a general formulation of binary mask based models for multi-domain learning by affine transformations of the original network parameters.
Open World Compositional Zero-Shot Learning
After estimating the feasibility score of each composition, we use these scores to either directly mask the output space or as a margin for the cosine similarity between visual features and compositional embeddings during training.
Towards Recognizing New Semantic Concepts in New Visual Domains
In the first part of the thesis, we describe different solutions to enable deep models to generalize to new visual domains, by transferring knowledge from a labeled source domain(s) to a domain (target) where no labeled data are available.
Prototype-based Incremental Few-Shot Semantic Segmentation
Semantic segmentation models have two fundamental weaknesses: i) they require large training sets with costly pixel-level annotations, and ii) they have a static output space, constrained to the classes of the training set.
Shape Consistent 2D Keypoint Estimation under Domain Shift
Recent unsupervised domain adaptation methods based on deep architectures have shown remarkable performance not only in traditional classification tasks but also in more complex problems involving structured predictions (e. g. semantic segmentation, depth estimation).
Towards Recognizing Unseen Categories in Unseen Domains
The key idea of CuMix is to simulate the test-time domain and semantic shift using images and features from unseen domains and categories generated by mixing up the multiple source domains and categories available during training.
Boosting Deep Open World Recognition by Clustering
While convolutional neural networks have brought significant advances in robot vision, their ability is often limited to closed world scenarios, where the number of semantic concepts to be recognized is determined by the available training set.
Modeling the Background for Incremental Learning in Semantic Segmentation
Current strategies fail on this task because they do not consider a peculiar aspect of semantic segmentation: since each training step provides annotation only for a subset of all possible classes, pixels of the background class (i. e. pixels that do not belong to any other classes) exhibit a semantic distribution shift.
Ranked #3 on
Domain 11-5
on Cityscapes
Knowledge is Never Enough: Towards Web Aided Deep Open World Recognition
While today's robots are able to perform sophisticated tasks, they can only act on objects they have been trained to recognize.
The RGB-D Triathlon: Towards Agile Visual Toolboxes for Robots
Deep networks have brought significant advances in robot perception, enabling to improve the capabilities of robots in several visual tasks, ranging from object detection and recognition to pose estimation, semantic scene segmentation and many others.
AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs
The ability to categorize is a cornerstone of visual intelligence, and a key functionality for artificial, autonomous visual machines.
Kitting in the Wild through Online Domain Adaptation
This novel dataset allows for testing the robustness of robot visual recognition algorithms to a series of different domain shifts both in isolation and unified.
Best sources forward: domain generalization through source-specific nets
A long standing problem in visual object categorization is the ability of algorithms to generalize across different testing conditions.
Ranked #122 on
Domain Generalization
on PACS
Robust Place Categorization with Deep Domain Generalization
Our method develops from the intuition that, given a set of different classification models associated to known domains (e. g. corresponding to multiple environments, robots), the best model for a new sample in the novel domain can be computed directly at test time by optimally combining the known models.
Adding New Tasks to a Single Network with Weight Transformations using Binary Masks
Visual recognition algorithms are required today to exhibit adaptive abilities.
Boosting Domain Adaptation by Discovering Latent Domains
Our approach is based on the introduction of two main components, which can be embedded into any existing CNN architecture: (i) a side branch that automatically computes the assignment of a source sample to a latent domain and (ii) novel layers that exploit domain membership information to appropriately align the distribution of the CNN internal feature representations to a reference distribution.
Inspiring Computer Vision System Solutions
Reviewing this project with modern eyes provides us with the opportunity to reflect on several issues, relevant now as then to the field of computer vision and research in general, that go beyond the technical aspects of the work.
Learning Deep NBNN Representations for Robust Place Categorization
This paper presents an approach for semantic place categorization using data obtained from RGB cameras.
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training
Word embeddings are widely used in Natural Language Processing, mainly due to their success in capturing semantic information from massive corpora.
