Search Results for author:
Found 42 papers, 14 papers with code
Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping
Developing agents that can execute multiple skills by learning from pre-collected datasets is an important problem in robotics, where online interaction with the environment is extremely time-consuming.
Fake it till you make it: Learning(s) from a synthetic ImageNet clone
We show that with minimal and class-agnostic prompt engineering those ImageNet clones we denote as ImageNet-SD are able to close a large part of the gap between models produced by synthetic images and models trained with real images for the several standard classification benchmarks that we consider in this study.
A soft nearest-neighbor framework for continual semi-supervised learning
Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data.
From CNNs to Shift-Invariant Twin Wavelet Models
We propose a novel antialiasing method to increase shift invariance in convolutional neural networks (CNNs).
Lightweight Structure-Aware Attention for Visual Understanding
Vision Transformers (ViTs) have become a dominant paradigm for visual representation learning with self-attention operators.
Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection
In defense-related remote sensing applications, such as vehicle detection on satellite imagery, supervised learning requires a huge number of labeled examples to reach operational performances.
Evaluating the Label Efficiency of Contrastive Self-Supervised Learning for Multi-Resolution Satellite Imagery
The application of deep neural networks to remote sensing imagery is often constrained by the lack of ground-truth annotations.
On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks
In this paper, we aim to improve the mathematical interpretability of convolutional neural networks for image classification.
No Reason for No Supervision: Improved Generalization in Supervised Models
We consider the problem of training a deep neural network on a given classification task, e. g., ImageNet-1K (IN1K), so that it excels at both the training task as well as at other (future) transfer tasks.
LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation
Recent works in autonomous driving have widely adopted the bird's-eye-view (BEV) semantic map as an intermediate representation of the world.
Walk the Random Walk: Learning to Discover and Reach Goals Without Supervision
Finally, we train a goal-conditioned policy network with goals sampled from the goal memory and reward it by the reachability network and the goal memory.
AVATAR: Unconstrained Audiovisual Speech Recognition
Audio-visual automatic speech recognition (AV-ASR) is an extension of ASR that incorporates visual cues, often from the movements of a speaker's mouth.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields
In this work, we argue that the coupling of camera rotation and camera translation can create complex motion fields that are difficult for a deep network to untangle directly.
Self-Supervised Models are Continual Learners
Self-supervised models have been shown to produce comparable or better visual representations than their supervised counterparts when trained offline on unlabeled data at scale.
Masking Modalities for Cross-modal Video Retrieval
Our proposal is to pre-train a video encoder using all the available video modalities as supervision, namely, appearance, sound, and transcribed speech.
Regularized Frank-Wolfe for Dense CRFs: Generalizing Mean Field and Beyond
We introduce regularized Frank-Wolfe, a general and effective algorithm for inference and learning of dense conditional random fields (CRFs).
Ranked #8 on
Semantic Segmentation
on Cityscapes test
LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR
In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e. g., with 64 beams, or camera-only methods, which suffer from scale-ambiguity and infinite-depth problems.
Memory-Augmented Reinforcement Learning for Image-Goal Navigation
In this work, we present a memory-augmented approach for image-goal navigation.
Context Aware Group Activity Recognition
Existing approaches decompose this task into feature learning and relational reasoning.
Dual-Tree Wavelet Packet CNNs for Image Classification
In this paper, we target an important issue of deep convolutional neural networks (CNNs) — the lack of a mathematical understanding of their properties.
Concept Generalization in Visual Representation Learning
In this paper, we argue that the semantic relationships between seen and unseen concepts affect generalization performance and propose ImageNet-CoG, a novel benchmark on the ImageNet-21K (IN-21K) dataset that enables measuring concept generalization in a principled way.
Meta-Learning with Shared Amortized Variational Inference
We propose a novel amortized variational inference scheme for an empirical Bayes meta-learning model, where model parameters are treated as latent variables.
The End-of-End-to-End: A Video Understanding Pentathlon Challenge (2020)
This report summarizes the results of the first edition of the challenge together with the findings of the participants.
Multi-modal Transformer for Video Retrieval
In this paper, we present a multi-modal transformer to jointly encode the different modalities in video, which allows each of them to attend to the others.
Ranked #14 on
Video Retrieval
on ActivityNet
(using extra training data)
Beyond the Camera: Neural Networks in World Coordinates
Eye movement and strategic placement of the visual field onto the retina, gives animals increased resolution of the scene and suppresses distracting information.
Meta-Learning by Hallucinating Useful Examples
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks, which aim to learn novel concepts from very few examples.
Adaptive Density Estimation for Generative Models
We show that our model significantly improves over existing hybrid models: offering GAN-like samples, IS and FID scores that are competitive with fully adversarial models, and improved likelihood scores.
Coverage and Quality Driven Training of Generative Image Models
First, we propose a model that extends variational autoencoders by using deterministic invertible transformation layers to map samples from the decoder to the image space.
How good is my GAN?
Generative adversarial networks (GANs) are one of the most popular methods for generating images today.
End-to-End Incremental Learning
Although deep learning approaches have stood out in recent years due to their state-of-the-art results, they continue to suffer from catastrophic forgetting, a dramatic decrease in overall performance when training with new classes added incrementally.
Ranked #2 on
Incremental Learning
on ImageNet - 10 steps
(# M Params metric)
Actor and Observer: Joint Modeling of First and Third-Person Videos
Several theories in cognitive neuroscience suggest that when people interact with the world, or simulate interactions, they do so from a first-person egocentric perspective, and seamlessly transfer knowledge between third-person (observer) and first-person (actor).
Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos
In this paper we describe the egocentric aspect of the dataset and present annotations for Charades-Ego with 68, 536 activity instances in 68. 8 hours of first and third-person video, making it one of the largest and most diverse egocentric datasets available.
Learning to Segment Moving Objects
We formulate this as a learning problem and design our framework with three cues: (i) independent object motion between a pair of frames, which complements object recognition, (ii) object appearance, which helps to correct errors in motion estimation, and (iii) temporal consistency, which imposes additional constraints on the segmentation.
Incremental Learning of Object Detectors without Catastrophic Forgetting
Despite their success for object detection, convolutional neural networks are ill-equipped for incremental learning, i. e., adapting the original model trained on a set of classes to additionally detect objects of new classes, in the absence of the initial training data.
Detecting Parts for Action Localization
In this paper, we propose a new framework for action localization that tracks people in videos and extracts full-body human tubes, i. e., spatio-temporal regions localizing actions, even in the case of occlusions or truncations.
Learning Video Object Segmentation with Visual Memory
The module to build a "visual memory" in video, i. e., a joint representation of all the video frames, is realized with a convolutional recurrent unit learned from a small number of training video sequences.
Learning Motion Patterns in Videos
The problem of determining whether an object is in motion, irrespective of camera motion, is far from being solved.
Weakly-Supervised Semantic Segmentation using Motion Cues
We also demonstrate that the performance of M-CNN learned with 150 weak video annotations is on par with state-of-the-art weakly-supervised methods trained with thousands of images.
Image Segmentation
Weakly supervised Semantic Segmentation
+1
Enhancing Energy Minimization Framework for Scene Text Recognition with Top-Down Cues
We build a conditional random field model on these detections to jointly model the strength of the detections and the interactions between them.
Online Object Tracking with Proposal Selection
Tracking-by-detection approaches are some of the most successful object trackers in recent years.
Mixing Body-Part Sequences for Human Pose Estimation
In this paper, we present a method for estimating articulated human poses in videos.
Learning to Estimate and Remove Non-uniform Image Blur
This paper addresses the problem of restoring images subjected to unknown and spatially varying blur caused by defocus or linear (say, horizontal) motion.
