Search Results for author:
Found 122 papers, 72 papers with code
Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI
The reconstruction of images observed by subjects from fMRI data collected during visual stimuli has made significant strides in the past decade, thanks to the availability of extensive fMRI datasets and advancements in generative models for image generation.
FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models
The pipeline is as follows: the image is passed to both a captioner model (i. e. BLIP) and a diffusion model (i. e., Stable Diffusion Model) to generate a text description and visual representation, respectively.
UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction
Vehicle trajectory prediction has increasingly relied on data-driven solutions, but their ability to scale to different data domains and the impact of larger dataset sizes on their generalization remain under-explored.
Ranked #1 on
Trajectory Prediction
on nuScenes
(using extra training data)
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs
The abilities of large language models (LLMs) have recently progressed to unprecedented levels, paving the way to novel applications in a wide variety of areas.
Manipulating Trajectory Prediction with Backdoors
In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction.
Reliability in Semantic Segmentation: Can We Use Synthetic Data?
Assessing the reliability of perception models to covariate shifts and out-of-distribution (OOD) detection is crucial for safety-critical applications such as autonomous vehicles.
ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation
Monocular 3D human pose estimation (3D-HPE) is an inherently ambiguous task, as a 2D pose in an image might originate from different possible 3D poses.
PointBeV: A Sparse Approach to BeV Predictions
To address this, we propose PointBeV, a novel sparse BeV segmentation model operating on sparse BeV cells instead of dense grids.
ToddlerDiffusion: Flash Interpretable Controllable Diffusion Model
Diffusion-based generative models excel in perceptually impressive synthesis but face challenges in interpretability.
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
Based on our ICL study, (3) we push ICL further and propose new multimodal ICL variants such as; Multitask-ICL, Chain-of-Hindsight-ICL, and Self-Correcting-ICL.
Gradpaint: Gradient-Guided Inpainting with Diffusion Models
For the specific task of image inpainting, the current guiding mechanism relies on copying-and-pasting the known regions from the input image at each denoising step.
DiffHPE: Robust, Coherent 3D Human Pose Lifting with Diffusion
We present an innovative approach to 3D Human Pose Estimation (3D-HPE) by integrating cutting-edge diffusion models, which have revolutionized diverse fields, but are relatively unexplored in 3D-HPE.
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks
Our model is efficiently pretrained on many tasks, based on task balancing and multimodal curriculum learning.
MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments
Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks for very large fully-annotated datasets.
Zero-shot spatial layout conditioning for text-to-image diffusion models
Large-scale text-to-image diffusion models have significantly improved the state of the art in generative image modelling and allow for an intuitive and powerful user interface to drive the image generation process.
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Large multimodal models trained on natural documents, which interleave images and text, outperform models trained on image-text pairs on various multimodal benchmarks.
Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?
In fact, conventional forecasting methods are usually not trained nor tested in real-world pipelines (e. g., with upstream detection, tracking, and mapping modules).
Improving Selective Visual Question Answering by Learning from Your Peers
In this work, we explore Selective VQA in both in-distribution (ID) and OOD scenarios, where models are presented with mixtures of ID and OOD data.
eP-ALM: Efficient Perceptual Augmentation of Language Models
In this work, we propose to rather direct effort to efficient adaptations of existing models, and propose to augment Language Models with perception.
PowerQuant: Automorphism Search for Non-Uniform Quantization
In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method.
Co-Training 2L Submodels for Visual Recognition
Given a neural network to be trained, for each sample we implicitly instantiate two altered networks, "submodels", with stochastic depth: i. e. activating only a subset of the layers and skipping others.
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization
In this paper, we thus propose model ratatouille, a new strategy to recycle the multiple fine-tunings of the same foundation model on diverse auxiliary tasks.
Ranked #14 on
Domain Generalization
on PACS
Co-training $2^L$ Submodels for Visual Recognition
We introduce submodel co-training, a regularization method related to co-training, self-distillation and stochastic depth.
Ranked #67 on
Image Classification
on ImageNet
Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
Finally, we validate the generalization of the approach to other tasks (i. e, Food Recognition) and domains with structured text such as the Medical domain on the ROCO dataset.
Ranked #1 on
Cross-Modal Retrieval
on Recipe1M+
CoMFormer: Continual Learning in Semantic and Panoptic Segmentation
%a In this paper, we present the first continual learning model capable of operating on both semantic and panoptic segmentation.
Ranked #2 on
Continual Semantic Segmentation
on ADE20K
OCTET: Object-aware Counterfactual Explanations
We conduct a set of experiments on counterfactual explanation benchmarks for driving scenes, and we show that our method can be adapted beyond classification, e. g., to explain semantic segmentation models.
DiffEdit: Diffusion-based semantic image editing with mask guidance
Semantic image editing is an extension of image generation, with the additional constraint that the generated image should be as similar as possible to a given input image.
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment
Vision and Language Pretraining has become the prevalent approach for tackling multimodal downstream tasks.
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance
The leap in performance in state-of-the-art computer vision methods is attributed to the development of deep neural networks.
LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation
Recent works in autonomous driving have widely adopted the bird's-eye-view (BEV) semantic map as an intermediate representation of the world.
Ranked #6 on
Bird's-Eye View Semantic Segmentation
on nuScenes
Dynamic Query Selection for Fast Visual Perceiver
Transformers have been matching deep convolutional networks for vision architectures in recent works.
Swapping Semantic Contents for Mixing Images
Deep architecture have proven capable of solving many tasks provided a sufficient amount of labeled data.
Towards efficient feature sharing in MIMO architectures
To solve this issue, we propose a novel unmixing step in MIMO architectures that allows subnetworks to properly share features.
Diverse Weight Averaging for Out-of-Distribution Generalization
Standard neural networks struggle to generalize under distribution shifts in computer vision.
Multi-Head Distillation for Continual Unsupervised Domain Adaptation in Semantic Segmentation
Unsupervised Domain Adaptation (UDA) is a transfer learning task which aims at training on an unlabeled target domain by leveraging a labeled source domain.
Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval
We propose a new retrieval framework, T-Food (Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval) that exploits the interaction between modalities in a novel regularization scheme, while using only unimodal encoders at test time for efficient retrieval.
Ranked #3 on
Cross-Modal Retrieval
on Recipe1M
DeiT III: Revenge of the ViT
Our evaluations on Image classification (ImageNet-1k with and without pre-training on ImageNet-21k), transfer learning and semantic segmentation show that our procedure outperforms by a large margin previous fully supervised training recipes for ViT.
Ranked #1 on
Image Classification
on ImageNet ReaL
(Number of params metric)
SPIQ: Data-Free Per-Channel Static Input Quantization
Computationally expensive neural networks are ubiquitous in computer vision and solutions for efficient inference have drawn a growing attention in the machine learning community.
Three things everyone should know about Vision Transformers
(2) Fine-tuning the weights of the attention layers is sufficient to adapt vision transformers to a higher resolution and to other classification tasks.
Ranked #8 on
Image Classification
on CIFAR-10
(using extra training data)
FlexIT: Towards Flexible Semantic Image Translation
Via the latent space of an auto-encoder, we iteratively transform the input image toward the target point, ensuring coherence and quality with a variety of novel regularization terms.
Augmenting Convolutional networks with attention-based aggregation
We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning.
Ranked #38 on
Semantic Segmentation
on ADE20K val
CSG0: Continual Urban Scene Generation with Zero Forgetting
With the rapid advances in generative adversarial networks (GANs), the visual quality of synthesised scenes keeps improving, including for complex urban scenes with applications to automated driving.
Embedding Arithmetic of Multimodal Queries for Image Retrieval
We introduce the SIMAT dataset to evaluate the task of Image Retrieval with Multimodal queries.
RED : Looking for Redundancies for Data-FreeStructured Compression of Deep Neural Networks
Deep Neural Networks (DNNs) are ubiquitous in today's computer vision landscape, despite involving considerable computational costs.
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion
Our strategy scales to a large number of tasks while having negligible memory and time overheads due to strict control of the parameters expansion.
Ranked #2 on
Incremental Learning
on ImageNet - 10 steps
STEEX: Steering Counterfactual Explanations with Semantics
In this work, we address the problem of producing counterfactual explanations for high-quality images and complex scenes.
Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
We describe a novel attribution method which is grounded in Sensitivity Analysis and uses Sobol indices.
RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging
Pruning Deep Neural Networks (DNNs) is a prominent field of study in the goal of inference runtime acceleration.
Effective Uncertainty Estimation with Evidential Models for Open-World Recognition
A crucial property of KLoS is to be a class-wise divergence measure built from in-distribution samples and to not require OOD training data, in contrast to current second-order uncertainty measures.
Raising context awareness in motion forecasting
Learning-based trajectory prediction models have encountered great success, with the promise of leveraging contextual information in addition to motion history.
LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR
In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e. g., with 64 beams, or camera-only methods, which suffer from scale-ambiguity and infinite-depth problems.
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization
In this paper, we introduce a new regularization - named Fishr - that enforces domain invariance in the space of the gradients of the loss: specifically, the domain-level variances of gradients are matched across training domains.
Ranked #22 on
Domain Generalization
on TerraIncognita
Multi-Target Adversarial Frameworks for Domain Adaptation in Semantic Segmentation
In this work, we address the task of unsupervised domain adaptation (UDA) for semantic segmentation in presence of multiple target domains: The objective is to train a single model that can handle all these domains at test time.
Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes.
Ranked #6 on
Overlapped 15-1
on PASCAL VOC 2012
Class Incremental Learning
Continual Semantic Segmentation
+5
Semantic Palette: Guiding Scene Generation with Class Proportions
Despite the recent progress of generative adversarial networks (GANs) at synthesizing photo-realistic images, producing complex urban scenes remains a challenging problem.
RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networks
Deep Neural Networks (DNNs) are ubiquitous in today's computer vision land-scape, despite involving considerable computational costs.
ResMLP: Feedforward networks for image classification with data-efficient training
We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification.
Ranked #1 on
Image Classification
on Certificate Verification
Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering
We use this new evaluation in a large-scale study of existing approaches for VQA.
Ranked #1 on
Visual Question Answering (VQA)
on VQA-CE
Going deeper with Image Transformers
In particular, we investigate the interplay of architecture and optimization of such dedicated transformers.
Ranked #5 on
Image Classification
on CIFAR-10
(using extra training data)
MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks
Recent strategies achieved ensembling "for free" by fitting concurrently diverse subnetworks inside a single base network.
Ranked #15 on
Image Classification
on Tiny ImageNet Classification
DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
Deep ensembles perform better than a single network thanks to the diversity among their members.
Explainability of deep vision-based autonomous driving systems: Review and challenges
The concept of explainability has several facets and the need for explainability is strong in driving, a safety-critical application.
Training data-efficient image transformers & distillation through attention
In this work, we produce a competitive convolution-free transformer by training on Imagenet only.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning
With this in mind, we propose a teacher-student scheme to learn representations by training a convolutional net to reconstruct a bag-of-visual-words (BoW) representation of an image, given as input a perturbed version of that same image.
Ranked #18 on
Semi-Supervised Image Classification
on ImageNet - 1% labeled data
(Top 5 Accuracy metric)
Confidence Estimation via Auxiliary Models
In this paper, we introduce a novel target criterion for model confidence, namely the true class probability (TCP).
Driving Behavior Explanation with Multi-level Fusion
In this era of active development of autonomous vehicles, it becomes crucial to provide driving systems with the capacity to explain their decisions.
Detecting 32 Pedestrian Attributes for Autonomous Vehicles
By increasing the number of attributes jointly learned, we highlight an issue related to the scales of gradients, which arises in MTL with numerous tasks.
Grafit: Learning fine-grained image representations with coarse labels
By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods.
Ranked #2 on
Learning with coarse labels
on cifar100
Fine-Grained Image Classification
Learning with coarse labels
+3
PLOP: Learning without Forgetting for Continual Semantic Segmentation
classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes.
Ranked #1 on
Domain 11-5
on Cityscapes val
Class Incremental Learning
Continual Semantic Segmentation
+16
Powers of layers for image-to-image translation
We propose a simple architecture to address unpaired image-to-image translation tasks: style or class transfer, denoising, deblurring, deblocking, etc.
Ranked #1 on
Image-to-Image Translation
on horse2zebra
(Frechet Inception Distance metric)
Insights from the Future for Continual Learning
Continual learning aims to learn tasks sequentially, with (often severe) constraints on the storage of old learning samples, without suffering from catastrophic forgetting.
Overcoming Statistical Shortcuts for Open-ended Visual Counting
First, we propose the Modifying Count Distribution (MCD) protocol, which penalizes models that over-rely on statistical shortcuts.
ESL: Entropy-guided Self-supervised Learning for Domain Adaptation in Semantic Segmentation
While fully-supervised deep learning yields good models for urban scene semantic segmentation, these models struggle to generalize to new environments with different lighting or weather conditions for instance.
PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning
Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning.
Deep Entwined Learning Head Pose and Face Alignment Inside an Attentional Cascade with Doubly-Conditional fusion
Head pose estimation and face alignment constitute a backbone preprocessing for many applications relying on face analysis.
Handling new target classes in semantic segmentation with domain adaptation
In this work, we define and address a novel domain adaptation (DA) problem in semantic scene segmentation, where the target domain not only exhibits a data distribution shift w. r. t.
Learning Representations by Predicting Bags of Visual Words
Inspired by the success of NLP methods in this area, in this work we propose a self-supervised approach based on spatially dense image descriptions that encode discrete visual concepts, here called visual words.
QUEST: Quantized embedding space for transferring knowledge
Knowledge distillation refers to the process of training a compact student network to achieve better accuracy by learning from a high capacity teacher network.
Addressing Failure Detection by Learning Model Confidence
In this paper, we propose a new target criterion for model confidence, corresponding to the True Class Probability (TCP).
RUBi: Reducing Unimodal Biases for Visual Question Answering
We propose RUBi, a new learning strategy to reduce biases in any VQA model.
This dataset does not exist: training models from generated images
This naturally brings the question: Can we train a classifier only on the generated data?
REVE: Regularizing Deep Learning with Variational Entropy Bound
Studies on generalization performance of machine learning algorithms under the scope of information theory suggest that compressed representations can guarantee good generalization, inspiring many compression-based regularization methods.
Addressing Failure Prediction by Learning Model Confidence
In this paper, we propose a new target criterion for model confidence, corresponding to the True Class Probability (TCP).
Riemannian batch normalization for SPD neural networks
Covariance matrices have attracted attention for machine learning applications due to their capacity to capture interesting structure in the data.
RUBi: Reducing Unimodal Biases in Visual Question Answering
We propose RUBi, a new learning strategy to reduce biases in any VQA model.
Ranked #7 on
Visual Question Answering (VQA)
on VQA-CP
Boosting Few-Shot Visual Learning with Self-Supervision
Few-shot learning and self-supervised learning address different facets of the same problem: how to train a model with little or no labeled data.
Zero-Shot Semantic Segmentation
Semantic segmentation models are limited in their ability to scale to large numbers of object classes.
Ranked #1 on
Zero-Shot Learning
on PASCAL Context
DualDis: Dual-Branch Disentangling with Adversarial Learning
To effectively separate the information, we propose to use a combination of regular and adversarial classifiers to guide the two branches in specializing for class and attribute information respectively.
The Missing Data Encoder: Cross-Channel Image Completion\\with Hide-And-Seek Adversarial Network
Image completion is the problem of generating whole images from fragments only.
SEMEDA: Enhancing Segmentation Precision with Semantic Edge Aware Loss
As a result, the predictions outputted by such networks usually struggle to accurately capture the object boundaries and exhibit holes inside the objects.
SoDeep: a Sorting Deep net to learn ranking loss surrogates
Our approach is based on a deep architecture that approximates the sorting of arbitrary sets of scores.
DeCaFA: Deep Convolutional Cascade for Face Alignment In The Wild
Face Alignment is an active computer vision domain, that consists in localizing a number of facial landmarks that vary across datasets.
Ranked #22 on
Face Alignment
on WFLW
DADA: Depth-aware Domain Adaptation in Semantic Segmentation
As a result, the performance of the trained semantic segmentation model on the target domain is boosted.
Ranked #17 on
Image-to-Image Translation
on SYNTHIA-to-Cityscapes
MUREL: Multimodal Relational Reasoning for Visual Question Answering
In this paper, we propose MuRel, a multimodal relational network which is learned end-to-end to reason over real images.
Ranked #1 on
Visual Question Answering (VQA)
on TDIUC
BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection
We demonstrate the practical interest of our fusion model by using BLOCK for two challenging tasks: Visual Question Answering (VQA) and Visual Relationship Detection (VRD), where we design end-to-end learnable architectures for representing relevant interactions between modalities.
Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection
Multi-Task Learning (MTL) is appealing for deep learning regularization.
ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
Semantic segmentation is a key problem for many computer vision tasks.
Ranked #4 on
Domain Adaptation
on Panoptic SYNTHIA-to-Mapillary
HybridNet: Classification and Reconstruction Cooperation for Semi-Supervised Learning
In this paper, we introduce a new model for leveraging unlabeled data to improve generalization performances of image classifiers: a two-branch encoder-decoder architecture called HybridNet.
Ranked #53 on
Image Classification
on STL-10
Manifold Learning in Quotient Spaces
Typically, we propose to quotient the space of 3D models by the action of rotations.
SHADE: Information-Based Regularization for Deep Learning
Regularization is a big issue for training deep neural networks.
Images & Recipes: Retrieval in the cooking context
Recent advances in the machine learning community allowed different use cases to emerge, as its association to domains like cooking which created the computational cuisine.
Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings
Designing powerful tools that support cooking activities has rapidly gained popularity due to the massive amounts of available data, as well as recent advances in machine learning that are capable of analyzing them.
Ranked #9 on
Cross-Modal Retrieval
on Recipe1M
SHADE: Information Based Regularization for Deep Learning
Regularization is a big issue for training deep neural networks.
Finding beans in burgers: Deep semantic-visual embedding with localization
Several works have proposed to learn a two-path neural network that maps images and texts, respectively, to a same shared Euclidean space where geometry captures useful semantic relationships.
GoSGD: Distributed Optimization for Deep Learning with Gossip Exchange
We address the issue of speeding up the training of convolutional neural networks by studying a distributed method adapted to stochastic gradient descent.
SHADE: SHAnnon DEcay Information-Based Regularization for Deep Learning
Regularization is a big issue for training deep neural networks.
Deformable Part-based Fully Convolutional Network for Object Detection
Existing region-based object detectors are limited to regions with fixed box geometry to represent objects, even if those are highly non-rectangular.
WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation
This paper introduces WILDCAT, a deep learning method which jointly aims at aligning image regions for gaining spatial invariance and learning strongly localized features.
Ranked #3 on
Weakly Supervised Object Detection
on MS COCO
MUTAN: Multimodal Tucker Fusion for Visual Question Answering
Bilinear models provide an appealing framework for mixing and merging information in Visual Question Answering (VQA) tasks.
Ranked #35 on
Visual Question Answering (VQA)
on VQA v2 test-std
Gossip training for deep learning
We address the issue of speeding up the training of convolutional networks.
Maxmin convolutional neural networks for image classification
Convolutional neural networks (CNN) are widely used in computer vision, especially in image classification.
Master's Thesis : Deep Learning for Visual Recognition
Our last contribution is a framework, build on top of Torch7, for training and testing deep models on any visual recognition tasks and on datasets of any scale.
M2CAI Workflow Challenge: Convolutional Neural Networks with Time Smoothing and Hidden Markov Model for Video Frames Classification
Our approach is among the three best to tackle the M2CAI Workflow challenge.
Closed-Form Training of Mahalanobis Distance for Supervised Clustering
Clustering is the task of grouping a set of objects so that objects in the same cluster are more similar to each other than to those in other clusters.
WELDON: Weakly Supervised Learning of Deep Convolutional Neural Networks
In this paper, we introduce a novel framework for WEakly supervised Learning of Deep cOnvolutional neural Networks (WELDON).
Deep Neural Networks Under Stress
In recent years, deep architectures have been used for transfer learning with state-of-the-art performance in many datasets.
MANTRA: Minimum Maximum Latent Structural SVM for Image Classification and Ranking
For ranking, we propose efficient solutions to exactly solve the inference and the loss-augmented problems.
Recipe recognition with large multimodal food dataset
We present deep experiments of recipe recognition on our dataset using visual, textual information and fusion.
Fantope Regularization in Metric Learning
This paper introduces a regularization method to explicitly control the rank of a learned symmetric positive semidefinite distance matrix in distance metric learning.
Sequentially Generated Instance-Dependent Image Representations for Classification
In this paper, we investigate a new framework for image classification that adaptively generates spatial representations.
Top-Down Regularization of Deep Belief Networks
We suggest a deep learning strategy that bridges the gap between the two phases, resulting in a three-phase learning procedure.
Dynamic Scene Classification: Learning Motion Descriptors with Slow Features Analysis
In this paper, we address the challenging problem of categorizing video sequences composed of dynamic natural scenes.
