Search Results for author:
Found 58 papers, 33 papers with code
Using body-anchored priors for identifying actions in single images
The task is easy for humans but difficult for current approaches to object recognition, because action instances may be similar in terms of body pose, and often require detailed examination of relations between participating objects and body parts in order to be recognized.
Fine-Grained Recognition of Thousands of Object Categories With Single-Example Training
We approach the problem of fast detection and recognition of a large number (thousands) of object categories while training on a very limited amount of examples, usually one per category.
RepMet: Representative-based metric learning for classification and one-shot object detection
Distance metric learning (DML) has been successfully applied to object classification, both in the standard regime of rich training data and in the few-shot scenario, where each category is represented by only a few examples.
Delta-encoder: an effective sample synthesis method for few-shot object recognition
Our approach is based on a modified auto-encoder, denoted Delta-encoder, that learns to synthesize new samples for an unseen category just by seeing few examples from it.
Co-regularized Alignment for Unsupervised Domain Adaptation
Deep neural networks, trained with large amount of labeled data, can fail to generalize well when tested with examples from a \emph{target domain} whose distribution differs from the training data distribution, referred as the \emph{source domain}.
LaSO: Label-Set Operations networks for multi-label few-shot learning
We conduct numerous experiments showing promising results for the label-set manipulation capabilities of the proposed approach, both directly (using the classification and retrieval metrics), and in the context of performing data augmentation for multi-label few-shot learning.
RepMet: Representative-Based Metric Learning for Classification and Few-Shot Object Detection
Distance metric learning (DML) has been successfully applied to object classification, both in the standard regime of rich training data and in the few-shot scenario, where each category is represented by only a few examples.
Baby steps towards few-shot learning with multiple semantics
Learning from one or few visual examples is one of the key capabilities of humans since early infancy, but is still a significant challenge for modern AI systems.
Ranked #9 on
Few-Shot Image Classification
on Mini-ImageNet - 1-Shot Learning
(using extra training data)
MetAdapt: Meta-Learned Task-Adaptive Architecture for Few-Shot Classification
In this work, we propose to employ tools inspired by the Differentiable Neural Architecture Search (D-NAS) literature in order to optimize the architecture for FSL without over-fitting.
A Broader Study of Cross-Domain Few-Shot Learning
Extensive experiments on the proposed benchmark are performed to evaluate state-of-art meta-learning approaches, transfer learning approaches, and newer methods for cross-domain few-shot learning.
Ranked #3 on
Cross-Domain Few-Shot
on Plantae
cross-domain few-shot learning
Few-Shot Image Classification
+1
TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classification
The field of Few-Shot Learning (FSL), or learning from very few (typically $1$ or $5$) examples per novel class (unseen during training), has received a lot of attention and significant performance advances in the recent literature.
StarNet: towards Weakly Supervised Few-Shot Object Detection
Few-shot detection and classification have advanced significantly in recent years.
OnlineAugment: Online Data Augmentation with Less Domain Knowledge
First is that most if not all modern augmentation search methods are offline and learning policies are isolated from their usage.
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition
Specifically, given a video frame, a policy network is used to decide what input resolution should be used for processing by the action recognition model, with the goal of improving both accuracy and efficiency.
Fine-grained Angular Contrastive Learning with Coarse Labels
A very practical example of C2FS is when the target classes are sub-classes of the training classes.
A Maximal Correlation Approach to Imposing Fairness in Machine Learning
As machine learning algorithms grow in popularity and diversify to many industries, ethical and legal concerns regarding their fairness have become increasingly relevant.
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
Temporal modelling is the key for efficient video action recognition.
Self-Supervised Classification Network
To guarantee non-degenerate solutions (i. e., solutions where all labels are assigned to the same class) we propose a mathematically motivated variant of the cross-entropy loss that has a uniform prior asserted on the predicted labels.
Ranked #3 on
Unsupervised Image Classification
on ImageNet
A Broad Study on the Transferability of Visual Representations with Contrastive Learning
Tremendous progress has been made in visual representation learning, notably with the recent success of self-supervised contrastive learning methods.
Detector-Free Weakly Supervised Grounding by Separation
In this work, we focus on the task of Detector-Free WSG (DF-WSG) to solve WSG without relying on a pre-trained detector.
Ranked #1 on
Phrase Grounding
on Visual Genome
Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data
As the base dataset and unlabeled dataset are from different domains, projecting the target images in the class-domain of the base dataset with a fixed pretrained model might be sub-optimal.
CHARTER: heatmap-based multi-type chart data extraction
The digital conversion of information stored in documents is a great source of knowledge.
Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data
It is thus better to tailor synthetic pre-training data to a specific downstream task, for best performance.
Unsupervised Domain Generalization by Learning a Bridge Across Domains
The ability to generalize learned representations across significantly different visual domains, such as between real photos, clipart, paintings, and sketches, is a fundamental capacity of the human visual system.
Task2Sim: Towards Effective Pre-Training and Transfer From Synthetic Data
It is thus better to tailor synthetic pre-training data to a specific downstream task, for best performance.
Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens
We explore a particular instantiation of scene structure, namely a \emph{Hand-Object Graph}, consisting of hands and objects with their locations as nodes, and physical relations of contact/no-contact as edges.
Structured Video Tokens @ Ego4D PNR Temporal Localization Challenge 2022
First, as both images and videos contain structured information, we enrich a transformer model with a set of \emph{object tokens} that can be used across images and videos.
Point- of-no-return (PNR) temporal localization
Temporal Localization
FETA: Towards Specializing Foundation Models for Expert Task Applications
However, as we show in this paper, FMs still have poor out-of-the-box performance on expert tasks (e. g. retrieval of car manuals technical illustrations from language queries), data for which is either unseen or belonging to a long-tail part of the data distribution of the huge datasets used for FM pre-training.
Ranked #1 on
Image-to-Text Retrieval
on FETA Car-Manuals
VL-Taboo: An Analysis of Attribute-based Zero-shot Capabilities of Vision-Language Models
We follow up with the analysis of the attribute-based zero-shot learning capabilities of these models, evaluating how well this classical zero-shot notion emerges from large-scale webly supervision.
Contrastive Audio-Visual Masked Autoencoder
In this paper, we first extend the recent Masked Auto-Encoder (MAE) model from a single modality to audio-visual multi-modalities.
Ranked #1 on
Audio Tagging
on AudioSet
(using extra training data)
C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval
Inspired by the fact that English text-video retrieval outperforms other languages, we train a student model using input text in different languages to match the cross-modal predictions from teacher models using input text in English.
On the Importance of Calibration in Semi-supervised Learning
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling.
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning
This leads to reasoning mistakes, which need to be corrected as they occur by teaching VL models the missing SVLC skills; often this must be done using private data where the issue was found, which naturally leads to a data-free continual (no task-id) VL learning setting.
Teaching Structured Vision&Language Concepts to Vision&Language Models
Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks.
On the Transferability of Visual Features in Generalized Zero-Shot Learning
Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features extracted from a pre-trained convolutional neural network.
CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning
Our experiments show that we outperform the current SOTA method DualPrompt on established benchmarks by as much as 4. 5% in average final accuracy.
MAEDAY: MAE for few and zero shot AnomalY-Detection
We propose using Masked Auto-Encoder (MAE), a transformer model self-supervisedly trained on image inpainting, for anomaly detection (AD).
PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data
In this work, we propose an approach to leverage synthetic scene data for improving video understanding.
Teaching Structured Vision & Language Concepts to Vision & Language Models
Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks.
Learning to Grow Pretrained Models for Efficient Transformer Training
Scaling transformers has led to significant breakthroughs in many domains, leading to a paradigm in which larger versions of existing models are trained and released on a periodic basis.
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks.
MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge
We adapt a VL model for zero-shot and few-shot action recognition using a collection of unlabeled videos and an unpaired action dictionary.
Ranked #3 on
Zero-Shot Action Recognition
on Kinetics
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
We contribute Synthetic Visual Concepts (SyViC) - a million-scale synthetic dataset and data generation codebase allowing to generate additional suitable data to improve VLC understanding and compositional reasoning of VL models.
Ranked #68 on
Visual Reasoning
on Winoground
Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies
Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned.
Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs
For the visual side, we incorporate a special "SG Component" in the image transformer trained to predict SG information, while for the textual side, we utilize SGs to generate fine-grained captions that highlight different compositional aspects of the scene.
Ranked #24 on
Visual Reasoning
on Winoground
Listen, Think, and Understand
On the other hand, modern large language models (LLMs) exhibit emerging reasoning ability but they lack audio perception capabilities.
Ranked #3 on
Music Question Answering
on MusicQA
(using extra training data)
Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages
Recent models such as XLS-R and Whisper have made multilingual speech technologies more accessible by pre-training on audio from around 100 spoken languages each.
TAP: Targeted Prompting for Task Adaptive Generation of Textual Training Instances for Visual Classification
Vision and Language Models (VLMs), such as CLIP, have enabled visual recognition of a potentially unlimited set of categories described by text prompts.
Joint Audio and Speech Understanding
Humans are surrounded by audio signals that include both speech and non-speech sounds.
Self-Specialization: Uncovering Latent Expertise within Large Language Models
Recent works have demonstrated the effectiveness of self-alignment in which a large language model is, by itself, aligned to follow general instructions through the automatic generation of instructional data using a handful of human-written seeds.
GeRA: Label-Efficient Geometrically Regularized Alignment
We introduce a semi-supervised Geometrically Regularized Alignment (GeRA) method to align the embedding spaces of pretrained unimodal encoders in a label-efficient way.
Learning Human Action Recognition Representations Without Real Humans
To this end, we present, for the first time, a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model.
3VL: using Trees to teach Vision & Language models compositional concepts
Vision-Language models (VLMs) have proved effective at aligning image and text representations, producing superior zero-shot results when transferred to many downstream tasks.
Large Scale Generative AI Text Applied to Sports and Music
We address the problem of scaling up the production of media content, including commentary and personalized news stories, for large-scale sports and music events worldwide.
CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory
Large Language Models (LLMs) struggle to handle long input sequences due to high memory and runtime costs.
Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs
Prompt ensembling of Large Language Model (LLM) generated category-specific prompts has emerged as an effective method to enhance zero-shot recognition ability of Vision-Language Models (VLMs).
Towards Multimodal In-Context Learning for Vision & Language Models
Inspired by the emergence of Large Language Models (LLMs) that can truly understand human language, significant progress has been made in aligning other, non-language, modalities to be `understandable' by an LLM, primarily via converting their samples into a sequence of embedded language-like tokens directly fed into the LLM (decoder) input stream.
NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning
Language models struggle with handling numerical data and performing arithmetic operations.
