Search Results for author:
Found 113 papers, 42 papers with code
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning
Since those regularization strategies are mostly associated with classifier outputs, we propose a MUlti-Classifier (MUC) incremental learning paradigm that integrates an ensemble of auxiliary classifiers to estimate more effective regularization constraints.
CrOC: Cross-View Online Clustering for Dense Visual Representation Learning
More importantly, the clustering algorithm conjointly operates on the features of both views, thereby elegantly bypassing the issue of content not represented in both views and the ambiguous matching of objects from one crop to the other.
Ranked #2 on
Unsupervised Semantic Segmentation
on COCO-Stuff
Adaptive Similarity Bootstrapping for Self-Distillation
Most self-supervised methods for representation learning leverage a cross-view consistency objective i. e. they maximize the representation similarity of a given image's augmented views.
Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning
Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building-block for many vision tasks.
Human-Object Interaction Detection
Knowledge Distillation
+2
Layout-aware Dreamer for Embodied Referring Expression Grounding
In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction.
Generative Negative Text Replay for Continual Vision-Language Pretraining
In this work, we focus on learning a VLP model with sequential chunks of image-text pair data.
Weakly Supervised Face Naming with Symmetry-Enhanced Contrastive Loss
We revisit the weakly supervised cross-modal face-name alignment task; that is, given an image and a caption, we label the faces in the image with the names occurring in the caption.
Students taught by multimodal teachers are superior action recognizers
Our approach is based on multimodal knowledge distillation, featuring a multimodal teacher (in the current experiments trained only using object detections, optical flow and RGB frames) and a unimodal student (using only RGB frames as input).
CLAD: A realistic Continual Learning benchmark for Autonomous Driving
In this paper we describe the design and the ideas motivating a new Continual Learning benchmark for Autonomous Driving (CLAD), that focuses on the problems of object classification and object detection.
FQDet: Fast-converging Query-based Detector
Recently, two-stage Deformable DETR introduced the query-based two-stage head, a new type of two-stage head different from the region-based two-stage heads of classical detectors as Faster R-CNN.
Category-Level Pose Retrieval with Contrastive Features Learnt with Occlusion Augmentation
In both cases, the idea is to directly predict the pose of an object.
Global-Local Self-Distillation for Visual Representation Learning
The downstream accuracy of self-supervised methods is tightly linked to the proxy task solved during training and the quality of the gradients extracted from it.
Continual evaluation for lifelong learning: Identifying the stability gap
Contemporary evaluation protocols and metrics in continual learning are task-based and quantify the trade-off between stability and plasticity only at task transitions.
Continual Pre-Training Mitigates Forgetting in Language and Vision
We formalize and investigate the characteristics of the continual pre-training scenario in both language and vision environments, where a model is continually pre-trained on a stream of incoming data and only later fine-tuned to different downstream tasks.
Re-examining Distillation For Continual Object Detection
Training models continually to detect and classify objects, from new classes and new domains, remains an open problem.
Spatial Consistency Loss for Training Multi-Label Classifiers from Single-Label Annotations
We show that adding a consistency loss, ensuring that the predictions of the network are consistent over consecutive training epochs, is a simple yet effective method to train multi-label classifiers in a weakly supervised setting.
New Insights on Reducing Abrupt Representation Change in Online Continual Learning
In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones.
Barlow constrained optimization for Visual Question Answering
In this paper, we propose a novel regularization for VQA models, Constrained Optimization using Barlow's theory (COB), that improves the information content of the joint space by minimizing the redundancy.
Find a Way Forward: a Language-Guided Semantic Map Navigator
In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semantic map.
RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues
In this work, we tackle this gap for the specific case of camera-based navigation, formulating it as following a visual cue in the foreground with arbitrary backgrounds.
Revisiting spatio-temporal layouts for compositional action recognition
Recognizing human actions is fundamentally a spatio-temporal reasoning problem, and should be, at least to some extent, invariant to the appearance of the human and the objects involved.
Ranked #31 on
Action Classification
on Charades
Trident Pyramid Networks: The importance of processing at the feature pyramid level for better object detection
Given their importance, a computer vision network can be divided into three parts: a backbone (generating a feature pyramid), a neck (refining the feature pyramid) and a head (generating the final output).
Towards Human-Understandable Visual Explanations: Human Imperceptible Cues Can Better Be Removed
Explainable AI (XAI) methods focus on explaining what a neural network has learned - in other words, identifying the features that are the most influential to the prediction.
Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling
We introduce a new task, unsupervised vision-language (VL) grammar induction.
Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration
In this paper, we propose the Glimpse-Attend-and-Explore model which: (a) employs self-attention to guide the visual exploration instead of task-specific uncertainty maps; (b) can be used for both dense and sparse prediction tasks; and (c) uses a contrastive stream to further improve the representations learned.
BlockCopy: High-Resolution Video Processing with Block-Sparse Feature Propagation and Online Policies
In this paper we propose BlockCopy, a scheme that accelerates pretrained frame-based CNNs to process video more efficiently, compared to standard frame-by-frame processing.
MinMaxCAM: Improving object coverage for CAM-basedWeakly Supervised Object Localization
One of the most common problems of weakly supervised object localization is that of inaccurate object coverage.
Weakly Supervised Object Localization
Weakly-Supervised Object Localization
Towards Human-Understandable Visual Explanations:Imperceptible High-frequency Cues Can Better Be Removed
Explainable AI (XAI) methods focus on explaining what a neural network has learned - in other words, identifying the features that are the most influential to the prediction.
Rehearsal revealed: The limits and merits of revisiting samples in continual learning
Learning from non-stationary data streams and overcoming catastrophic forgetting still poses a serious challenge for machine learning research.
New Insights on Reducing Abrupt Representation Change in Online Continual Learning
In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones.
Avalanche: an End-to-End Library for Continual Learning
Learning continually from non-stationary data streams is a long-standing goal and a challenging problem in machine learning.
SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
For instance, our method reduces the number of floating-point operations of SwiftNet-RN18 by 60% and increases the inference speed by 50%, with only 0. 3% decrease in mIoU accuracy on Cityscapes.
Ranked #5 on
Semantic Segmentation
on Mapillary val
Decoding Language Spatial Relations to 2D Spatial Arrangements
We address the problem of multimodal spatial understanding by decoding a set of language-expressed spatial relations to a set of 2D spatial arrangements in a multi-object and multi-relationship setting.
Learning to ground medical text in a 3D human atlas
In this paper, we develop a method for grounding medical text into a physically meaningful and interpretable space corresponding to a human atlas.
Can the state of relevant neurons in a deep neural networks serve as indicators for detecting adversarial attacks?
We present a method for adversarial attack detection based on the inspection of a sparse set of neurons.
On the Exploration of Incremental Learning for Fine-grained Image Retrieval
On the other hand, fine-tuning the learned representation only with the new classes leads to catastrophic forgetting.
Self-Supervised Ranking for Representation Learning
We present a new framework for self-supervised representation learning by formulating it as a ranking problem in an image retrieval context on a large number of random views (augmentations) obtained from images.
Feed-Forward On-Edge Fine-tuning Using Static Synthetic Gradient Modules
By first training the SGMs in a meta-learning manner on a set of common objects, during fine-tuning, the SGMs provided the model with accurate gradients to successfully learn to grasp new objects.
Commands 4 Autonomous Vehicles (C4AV) Workshop Summary
In this work, we deviate from recent, popular task settings and consider the problem under an autonomous vehicle scenario.
Ranked #2 on
Referring Expression Comprehension
on Talk2Car
Multiple Exemplars-based Hallucinationfor Face Super-resolution and Editing
Given a really low-resolution input image of a face (say 16x16 or 8x8 pixels), the goal of this paper is to reconstruct a high-resolution version thereof.
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams
Attaining prototypical features to represent class distributions is well established in representation learning.
What My Motion tells me about Your Pose: A Self-Supervised Monocular 3D Vehicle Detector
We subsequently demonstrate an optimization-based monocular 3D bounding box detector built on top of the self-supervised vehicle orientation estimator without the requirement of expensive labeled data.
Attend and Segment: Attention Guided Active Semantic Segmentation
The main idea is to refine an agent's understanding of the environment by attending the areas it is most uncertain about.
MIX'EM: Unsupervised Image Classification using a Mixture of Embeddings
We introduce three techniques to successfully train MIX'EM and avoid degenerate solutions; (i) diversify mixture components by maximizing entropy, (ii) minimize instance conditioned component entropy to enforce a clustered embedding space, and (iii) use an associative embedding loss to enforce semantic separability.
Self-supervised context-aware COVID-19 document exploration through atlas grounding
In this paper, we aim to develop a self-supervised grounding of Covid-related medical text based on the actual spatial relationships between the referred anatomical concepts.
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity.
Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem
This framework flexibly disentangles user-adaptation into model personalization on the server and local data regularization on the user device, with desirable properties regarding scalability and privacy constraints.
Deep-Geometric 6 DoF Localization from a Single Image in Topo-metric Maps
We describe a Deep-Geometric Localizer that is able to estimate the full 6 Degree of Freedom (DoF) global pose of the camera from a single image in a previously mapped environment.
Information Compensation for Deep Conditional Generative Networks
In recent years, unsupervised/weakly-supervised conditional generative adversarial networks (GANs) have achieved many successes on the task of modeling and generating data.
Ternary Feature Masks: zero-forgetting for task-incremental learning
To allow already learned features to adapt to the current task without changing the behavior of these features for previous tasks, we introduce task-specific feature normalization.
Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference
Modern convolutional neural networks apply the same operations on every pixel in an image.
Mixture Dense Regression for Object Detection and Human Pose Estimation
We realize the framework for object detection and human pose estimation.
Online Continual Learning with Maximal Interfered Retrieval
Methods based on replay, either generative or from a stored memory, have been shown to be effective approaches for continual learning, matching or exceeding the state of the art in a number of standard benchmarks.
Where to Look Next: Unsupervised Active Visual Exploration on 360° Input
We address the problem of active visual exploration of large 360{\deg} inputs.
How to improve CNN-based 6-DoF camera pose estimation
Convolutional neural networks (CNNs) and transfer learning have recently been used for 6 degrees of freedom (6-DoF) camera pose estimation.
A continual learning survey: Defying forgetting in classification tasks
Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase.
In Defense of LSTMs for Addressing Multiple Instance Learning Problems
LSTMs have a proven track record in analyzing sequential data.
Online Continual Learning with Maximally Interfered Retrieval
Methods based on replay, either generative or from a stored memory, have been shown to be effective approaches for continual learning, matching or exceeding the state of the art in a number of standard benchmarks.
Exploring the Challenges towards Lifelong Fact Learning
So far life-long learning (LLL) has been studied in relatively small-scale and relatively artificial setups.
Task-Free Continual Learning
A sequence of tasks is learned, one at a time, with all data of current task available but not of previous or future tasks.
An Unpaired Shape Transforming Method for Image Translation and Cross-Domain Retrieval
We address the problem of unpaired geometric image-to-image translation.
Selfless Sequential Learning
In particular, we propose a novel regularizer, that encourages representation sparsity by means of neural inhibition.
Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency
Experimental results on various datasets show that EGSC-IT does not only translate the source image to diverse instances in the target domain, but also preserves the semantic consistency during the process.
Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks
In this paper, we propose a novel scheme for both interpretation as well as explanation in which, given a pretrained model, we automatically identify internal features relevant for the set of classes considered by the model, without relying on additional annotations.
Super-Resolution with Deep Adaptive Image Resampling
In this work, we revisit the more traditional interpolation-based methods, that were popular before, now with the help of deep learning.
Error Correction for Dense Semantic Image Labeling
Pixelwise semantic image labeling is an important, yet challenging, task with many applications.
Memory Aware Synapses: Learning what (not) to forget
We show state-of-the-art performance and, for the first time, the ability to adapt the importance of the parameters based on unlabeled data towards what the network needs (not) to forget, which may vary depending on test conditions.
An Analysis of Human-centered Geolocation
Experiments on the Fashion 144k and a Pinterest-based dataset show that the automatic methods succeed at this task to a reasonable extent.
Pose Guided Person Image Generation
This paper proposes the novel Pose Guided Person Generation Network (PG$^2$) that allows to synthesize person images in arbitrary poses, based on an image of that person and a novel pose.
Ranked #3 on
Gesture-to-Gesture Translation
on Senz3D
Speech-Based Visual Question Answering
This paper introduces speech-based visual question answering (VQA), the task of generating an answer given an image and a spoken question.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Context-based Object Viewpoint Estimation: A 2D Relational Approach
To estimate the viewpoint (or pose) of an object, people have mostly looked at object intrinsic features, such as shape or appearance.
Encoder Based Lifelong Learning
This paper introduces a new lifelong learning solution where a single model is trained for a sequence of tasks.
Learning to Recognize Animals by Watching Documentaries: Using Subtitles as Weak Supervision
We investigate animal recognition models learned from wildlife video documentaries by using the weak supervision of the textual subtitles.
How hard is it to cross the room? -- Training (Recurrent) Neural Networks to steer a UAV
To cope with more complex tasks, we propose the use of recurrent neural networks (RNN) instead and successfully train an LSTM (Long-Short Term Memory) network for controlling UAVs.
Who's that Actor? Automatic Labelling of Actors in TV series starting from IMDB Images
In this work, we aim at automatically labeling actors in a TV series.
What Is Around The Camera?
How much does a single image reveal about the environment it was taken in?
Expert Gate: Lifelong Learning with a Network of Experts
Further, the autoencoders inherently capture the relatedness of one task to another, based on which the most relevant prior model to be used for training a new expert, with finetuning or learning without-forgetting, can be selected.
Reasoning about Body-Parts Relations for Sign Language Recognition
Over the years, hand gesture recognition has been mostly addressed considering hand trajectories in isolation.
DeepProposals: Hunting Objects and Actions by Cascading Deep Convolutional Layers
In this paper, a new method for generating object and action proposals in images and videos is proposed.
Dynamic Filter Networks
In a traditional convolutional layer, the learned filters stay fixed after training.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Online Action Detection
Third, the start of the action is unknown, so it is unclear over what time window the information should be integrated.
Modeling Visual Compatibility through Hierarchical Mid-level Elements
At the base-level, our method identifies patterns of CNN activations with the aim of modeling different variations/styles in which objects of the classes of interest may occur.
Cross-modal Supervision for Learning Active Speaker Detection in Video
We further improve a generic model for active speaker detection by learning person specific models.
DeLight-Net: Decomposing Reflectance Maps into Specular Materials and Natural Illumination
In this paper we are extracting surface reflectance and natural environmental illumination from a reflectance map, i. e. from a single 2D image of a sphere of one material under one illumination.
Unsupervised Domain Adaptation in the Wild: Dealing with Asymmetric Label Sets
Most methods for unsupervised domain adaptation proposed in the literature to date, assume that the set of classes present in the target domain is identical to the set of classes present in the source domain.
Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction
To this end, we first analyze the output of each convolutional layer from a domain adaptation perspective.
Novel Views of Objects from a Single Image
We propose a technique to use the structural information extracted from a 3D model that matches the image object in terms of viewpoint and shape.
Rank Pooling for Action Recognition
We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation.
Continuous Pose Estimation With a Spatial Ensemble of Fisher Regressors
In this paper, we treat the problem of continuous pose estimation for object categories as a regression problem on the basis of only 2D training information.
Learning to Rank Based on Subsequences
We present a supervised learning to rank algorithm that effectively orders images by exploiting the structure in image sequences.
Guiding the Long-Short Term Memory Model for Image Caption Generation
In this work we focus on the problem of image caption generation.
Learning Where to Position Parts in 3D
In this paper we propose a new method for the detection and pose estimation of 3D objects, that does not use any 3D CAD model or other 3D information.
MidRank: Learning to rank based on subsequences
We present a supervised learning to rank algorithm that effectively orders images by exploiting the structure in image sequences.
Towards Automatic Image Editing: Learning to See another You
Learning the distribution of images in order to generate new samples is a challenging task due to the high dimensionality of the data and the highly non-linear relations that are involved.
Deep Reflectance Maps
Undoing the image formation process and therefore decomposing appearance into its intrinsic properties is a challenging task due to the under-constraint nature of this inverse problem.
Recovering hard-to-find object instances by sampling context-based object proposals
In this paper we focus on improving object detection performance in terms of recall.
DeepProposal: Hunting Objects by Cascading Deep Convolutional Layers
We generate hypotheses in a sliding-window fashion over different activation layers and show that the final convolutional layers can find the object of interest with high recall but poor localization due to the coarseness of the feature maps.
Active Transfer Learning with Zero-Shot Priors: Reusing Past Datasets for Future Tasks
How can we reuse existing knowledge, in the form of available datasets, when solving a new and apparently unrelated target task from a set of unlabeled data?
Guiding Long-Short Term Memory for Image Caption Generation
In this work we focus on the problem of image caption generation.
Subspace Alignment Based Domain Adaptation for RCNN Detector
In this paper, we propose subspace alignment based domain adaptation of the state of the art RCNN based object detector.
Modeling Video Evolution for Action Recognition
We postulate that a function capable of ordering the frames of a video temporally (based on the appearance) captures well the evolution of the appearance within the video.
Dataset Fingerprints: Exploring Image Collections Through Data Mining
As the amount of visual data increases, so does the need for summarization tools that can be used to explore large image collections and to quickly get familiar with their content.
Weakly Supervised Object Detection With Convex Clustering
However, as learning appearance and localization are two interconnected tasks, the optimization is not convex and the procedure can easily get stuck in a poor local minimum, the algorithm "misses" the object in some images.
A Deeper Look at Dataset Bias
The presence of a bias in each image data collection has recently attracted a lot of attention in the computer vision community showing the limits in generalization of any learning method trained on a specific dataset.
Mind the Gap: Subspace based Hierarchical Domain Adaptation
Domain adaptation techniques aim at adapting a classifier learnt on a source domain to work on the target domain.
Joint cross-domain classification and subspace learning for unsupervised adaptation
Domain adaptation aims at adapting the knowledge acquired on a source domain to a new different but related target domain.
Location Recognition Over Large Time Lags
Would it be possible to automatically associate ancient pictures to modern ones and create fancy cultural heritage city maps?
Subspace Alignment For Domain Adaptation
We present two approaches to determine the only hyper-parameter in our method corresponding to the size of the subspaces.
Image-based Synthesis and Re-Synthesis of Viewpoints Guided by 3D Models
We propose a technique to use the structural information extracted from a set of 3D models of an object class to improve novel-view synthesis for images showing unknown instances of this class.
Using a Deformation Field Model for Localizing Faces and Facial Points under Weak Supervision
Additionally, without any facial point annotation at the level of individual training images, our method can localize facial points with an accuracy similar to fully supervised approaches.
Object Classification with Adaptable Regions
In classification of objects substantial work has gone into improving the low level representation of an image by considering various aspects such as different features, a number of feature pooling and coding techniques and considering different kernels.
A Testbed for Cross-Dataset Analysis
Since its beginning visual recognition research has tried to capture the huge variability of the visual world in several image collections.
Seeking the Strongest Rigid Detector
The current state of the art solutions for object detection describe each class by a set of models trained on discovered sub-classes (so called "components"), with each model itself composed of collections of interrelated parts (deformable models).
