Search Results for author:
Found 197 papers, 87 papers with code
Flex3D: Feed-Forward 3D Generation With Flexible Reconstruction Model And Input View Curation
We employ a fine-tuned multi-view image diffusion model and a video diffusion model to generate a pool of candidate views, enabling a rich representation of the target 3D object.
CatFree3D: Category-agnostic 3D Object Detection with Diffusion
Image-based 3D object detection is widely employed in applications such as autonomous vehicles and robotics, yet current systems struggle with generalisation due to complex problem setup and limited training data.
3D-Aware Instance Segmentation and Tracking in Egocentric Videos
Egocentric videos present unique challenges for 3D scene understanding due to rapid camera motion, frequent object occlusions, and limited object visibility.
Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics
We present Puppet-Master, an interactive video generative model that can serve as a motion prior for part-level dynamics.
SHIC: Shape-Image Correspondences with no Keypoint Supervision
The reduction works by matching images of the object to non-photorealistic renders of the template, which emulates the process of collecting manual annotations for this task.
Meta 3D Gen
We introduce Meta 3D Gen (3DGen), a new state-of-the-art, fast pipeline for text-to-3D asset generation.
Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objects
The recent availability and adaptability of text-to-image models has sparked a new era in many related domains that benefit from the learned text priors as well as high-quality and fast generation capabilities, one of which is texture generation for 3D objects.
Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials
We present Meta 3D AssetGen (AssetGen), a significant advancement in text-to-3D generation which produces faithful, high-quality meshes with texture and material control.
Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image
In this paper, we propose Flash3D, a method for scene reconstruction and novel view synthesis from a single image which is both very generalisable and efficient.
Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting
These works often depend on pre-trained monocular depth estimators to lift the generated images into 3D, fusing them with the existing scene representation.
Lightplane: Highly-Scalable Components for Neural 3D Fields
Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision.
DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing
A common approach to this problem is to use a 2D image generator or editor to guide the 3D editing process, obviating the need for 3D data.
DragAPart: Learning a Part-Level Motion Prior for Articulated Objects
We introduce DragAPart, a method that, given an image and a set of drags as input, generates a new image of the same object that responds to the action of the drags.
N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fields
To address this, we introduce Nested Neural Feature Fields (N2F2), a novel approach that employs hierarchical supervision to learn a single feature field, wherein different dimensions within the same high-dimensional feature encode scene properties at varying granularities.
GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering
With the aid of a frequency-modulated loss, GES achieves competitive performance in novel-view synthesis benchmarks while requiring less than half the memory storage of Gaussian Splatting and increasing the rendering speed by up to 39%.
IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation
A mitigation is to fine-tune the 2D generator to be multi-view aware, which can help distillation or can be combined with reconstruction networks to output 3D objects directly.
Learning the 3D Fauna of the Web
We show that prior category-specific attempts fail to generalize to rare species with limited training images.
GES : Generalized Exponential Splatting for Efficient Radiance Field Rendering
With the aid of a frequency-modulated loss GES achieves competitive performance in novel-view synthesis benchmarks while requiring less than half the memory storage of Gaussian Splatting and increasing the rendering speed by up to 39%.
Splatter Image: Ultra-Fast Single-View 3D Reconstruction
Splatter Image is based on Gaussian Splatting, which allows fast and high-quality reconstruction of 3D scenes from multiple images.
GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning
Encoding information from 2D views of an object into a 3D representation is crucial for generalized 3D feature extraction.
SHAP-EDITOR: Instruction-guided Latent 3D Editing in Seconds
In particular, we hypothesise that editing can be greatly simplified by first encoding 3D objects in a suitable latent space.
Free3D: Consistent Novel View Synthesis without 3D Representation
Similar to Zero-1-to-3, we start from a pre-trained 2D image generator for generalization, and fine-tune it for NVS.
Instant Uncertainty Calibration of NeRFs Using a Meta-Calibrator
To address this, we introduce the concept of a meta-calibrator that performs uncertainty calibration for NeRFs with a single forward pass without the need for holding out any images from the target scene.
HoloFusion: Towards Photo-realistic 3D Generative Modeling
Diffusion-based image generators can now produce high-quality and diverse samples, but their success has yet to fully translate to 3D generation: existing diffusion methods can either generate low-resolution but 3D consistent outputs, or detailed 2D views of 3D objects but with potential structural defects and lacking view consistency or realism.
Online Clustered Codebook
Vector Quantisation (VQ) is experiencing a comeback in machine learning, where it is increasingly used in representation learning.
Replay: Multi-modal Multi-view Acted Videos for Casual Holography
We introduce Replay, a collection of multi-view, multi-modal videos of humans interacting socially.
CoTracker: It is Better to Track Together
We introduce CoTracker, a transformer-based model that tracks a large number of 2D points in long video sequences.
Ranked #2 on
Point Tracking
on TAP-Vid-Kinetics-First
Diffusion Models for Open-Vocabulary Segmentation
Open-vocabulary segmentation is the task of segmenting anything that can be named in an image.
EPIC Fields: Marrying 3D Geometry and Video Understanding
Compared to other neural rendering datasets, EPIC Fields is better tailored to video understanding because it is paired with labelled action segments and the recent VISOR segment annotations.
Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data
We fit a diffusion model to a large number of viewsets for a given category of objects.
Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion
Our approach outperforms the state-of-the-art on challenging scenes from the ScanNet, Hypersim, and Replica datasets, as well as on our newly created Messy Rooms dataset, demonstrating the effectiveness and scalability of our slow-fast clustering method.
DynamicStereo: Consistent Dynamic Depth from Stereo Videos
The network learns to pool information from neighboring frames to improve the temporal consistency of its predictions.
Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion
We propose a framework that uses an image generator, such as Stable Diffusion, to generate synthetic training data that are sufficiently clean and do not require further manual curation, enabling the learning of such a reconstruction network from scratch.
What does CLIP know about a red circle? Visual prompt engineering for VLMs
Large-scale Vision-Language Models, such as CLIP, learn powerful image-text representations that have found numerous applications, from zero-shot classification to text-to-image generation.
Continual Detection Transformer for Incremental Object Detection
Incremental object detection (IOD) aims to train an object detector in phases, each with annotations for new object categories.
Class-Incremental Object Detection
Knowledge Distillation
+3
Training-Free Layout Control with Cross-Attention Guidance
We thoroughly evaluate our approach on three benchmarks and provide several qualitative examples and a comparative analysis of the two strategies that demonstrate the superiority of backward guidance compared to forward guidance, as well as prior work.
HoloDiffusion: Training a 3D Diffusion Model using 2D Images
We show that our diffusion models are scalable, train robustly, and are competitive in terms of sample quality and fidelity to existing approaches for 3D generative modeling.
Real-time volumetric rendering of dynamic humans
We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits.
$PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction
Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision.
RealFusion: 360° Reconstruction of Any Object from a Single Image
We consider the problem of reconstructing a full 360{\deg} photographic model of an object from a single image of it.
Text-To-4D Dynamic Scene Generation
We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions.
Novel-View Acoustic Synthesis
We introduce the novel-view acoustic synthesis (NVAS) task: given the sight and sound observed at a source viewpoint, can we synthesize the sound of that scene from an unseen target viewpoint?
PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction
Reconstructing the 3D shape of an object from a single RGB image is a long-standing problem in computer vision.
RealFusion: 360deg Reconstruction of Any Object From a Single Image
We consider the problem of reconstructing a full 360deg photographic model of an object from a single image of it.
Self-Supervised Correspondence Estimation via Multiview Registration
To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences.
MagicPony: Learning Articulated 3D Animals in the Wild
We consider the problem of predicting the 3D shape, articulation, viewpoint, texture, and lighting of an articulated animal like a horse given a single test image as input.
Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories
Obtaining photorealistic reconstructions of objects from sparse views is inherently ambiguous and can only be achieved by learning suitable reconstruction priors.
Unsupervised Multi-object Segmentation by Predicting Probable Motion Patterns
We propose a new approach to learn to segment multiple image objects without manual supervision.
Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing
Self-supervised visual representation learning has recently attracted significant research interest.
Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations
We present Neural Feature Fusion Fields (N3F), a method that improves dense 2D image feature extractors when the latter are applied to the analysis of multiple images reconstructible as a 3D scene.
SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data
We present a method for the accurate 3D reconstruction of partly-symmetric objects.
Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion
Motion, measured via optical flow, provides a powerful cue to discover and learn objects in images and videos.
Ranked #4 on
Unsupervised Object Segmentation
on SegTrack-v2
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization
We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene.
End-to-End Visual Editing with a Generatively Pre-Trained Artist
We consider the targeted image editing problem: blending a region in a source image with a driver image that specifies the desired change.
Generalized Category Discovery
Here, the unlabelled images may come from labelled classes or from novel ones.
Ranked #1 on
Open-World Semi-Supervised Learning
on CIFAR-10
(Seen accuracy (50% Labeled) metric)
Fine-Grained Visual Recognition
Open-World Semi-Supervised Learning
+1
KeyTr: Keypoint Transporter for 3D Reconstruction of Deformable Objects in Videos
Compared to weaker deformation models, this significantly reduces the reconstruction ambiguity and, for dynamic objects, allows Keypoint Transporter to obtain reconstructions of the quality superior or at least comparable to prior approaches while being much faster and reliant on a pre-trained monocular depth estimator network.
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model.
Audio-Visual Synchronisation in the wild
Finally, we set the first benchmark for general audio-visual synchronisation with over 160 diverse classes in the new VGG-Sound Sync video dataset.
Unsupervised Part Discovery from Contrastive Reconstruction
First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts.
Ranked #1 on
Unsupervised Keypoint Estimation
on CUB
The Curious Layperson: Fine-Grained Image Recognition without Expert Labels
We then train a fine-grained textual similarity model that matches image descriptions with documents on a sentence-level basis.
NeuralDiff: Segmenting 3D objects that move in egocentric videos
Given a raw video sequence taken from a freely-moving camera, we study the problem of decomposing the observed 3D scene into a static background and a dynamic foreground containing the objects that move in the video sequence.
Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
In this paper, we first demonstrate that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes.
Ranked #10 on
Out-of-Distribution Detection
on CIFAR-100 vs CIFAR-10
Measuring the Interpretability of Unsupervised Representations via Quantized Reversed Probing
Self-supervised visual representation learning has attracted significant research interest.
Ranked #90 on
Image Classification
on ObjectNet
(using extra training data)
PASS: An ImageNet replacement for self-supervised pretraining without humans
On the other hand, state-of-the-art pretraining is nowadays obtained with unsupervised methods, meaning that labelled datasets such as ImageNet may not be necessary, or perhaps not even optimal, for model pretraining.
Lifting 2D Object Locations to 3D by Discounting LiDAR Outliers across Objects and Views
We present a system for automatic converting of 2D mask object predictions and raw LiDAR point clouds into full 3D bounding boxes of objects.
Real Time Monocular Vehicle Velocity Estimation using Synthetic Data
Vision is one of the primary sensing modalities in autonomous driving.
DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension
The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object.
Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields
Implicit neural representation is a recent approach to learn shape collections as zero level-sets of neural networks, where each shape is represented by a latent code.
DOVE: Learning Deformable 3D Objects by Watching Videos
In this paper, we present DOVE, a method that learns textured 3D models of deformable object categories from monocular videos available online, without keypoint, viewpoint or template shape supervision.
AutoNovel: Automatically Discovering and Learning Novel Visual Categories
We present a new approach called AutoNovel to address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labelled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use ranking statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data.
Ranked #1 on
Novel Class Discovery
on SVHN
Pedestrian and Ego-Vehicle Trajectory Prediction From Monocular Camera
Predicting future pedestrian trajectory is a crucial component of autonomous driving systems, as recognizing critical situations based only on current pedestrian position may come too late for any meaningful corrective action (e. g. breaking) to take place.
NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go
We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes and produces in one go, i. e. in a single feed forward pass, a smooth interpolation and point-to-point correspondences between them.
Discovering Relationships between Object Categories via Universal Canonical Maps
Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects.
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in representation space.
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers
In video transformers, the time dimension is often treated in the same way as the two spatial dimensions.
Ranked #15 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
Recent research has shown that numerous human-interpretable directions exist in the latent space of GANs.
Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes
We build on the idea of view synthesis, which uses classical camera geometry to re-render a source image from a different point-of-view, specified by a predicted relative pose and depth map.
Self-supervised object detection from audio-visual correspondence
We tackle the problem of learning object detectors without supervision.
Localizing Visual Sounds the Hard Way
We show that our algorithm achieves state-of-the-art performance on the popular Flickr SoundNet dataset.
Unsupervised Learning of 3D Object Categories from Videos in the Wild
Our goal is to learn a deep network that, given a small number of images of an object of a given category, reconstructs it in 3D.
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning
First, for space, we show that spatial augmentations such as cropping do work well for videos too, but that previous implementations, due to the high processing and memory cost, could not do this at a scale sufficient for it to work well.
Continuous Surface Embeddings
In this work, we focus on the task of learning and representing dense correspondences in deformable object categories.
3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data
We consider the problem of obtaining dense 3D reconstructions of humans from single and partially occluded views.
Ranked #3 on
Multi-Hypotheses 3D Human Pose Estimation
on AH36M
Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
The increasing impact of black box models, and particularly of unsupervised ones, comes with an increasing interest in tools to understand and interpret them.
Support-set bottlenecks for video-text representation learning
The dominant paradigm for learning video-text representations -- noise contrastive learning -- increases the similarity of the representations of pairs of samples that are known to be related, such as text and video from the same sample, and pushes away the representations of all other pairs.
Multi-modal Self-Supervision from Generalized Data Transformations
In this paper, we show that, for videos, the answer is more complex, and that better results can be obtained by accounting for the interplay between invariance, distinctiveness, multiple modalities and time.
Calibrating Self-supervised Monocular Depth Estimation
In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal.
Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstruction
We propose the Canonical 3D Deformer Map, a new representation of the 3D shape of common object categories that can be learned from a collection of 2D images of independent objects.
RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces
We present RELATE, a model that learns to generate physically plausible scenes and videos of multiple interacting objects.
Labelling unlabelled videos from scratch with multi-modal self-supervision
A large part of the current success of deep learning lies in the effectiveness of data -- more precisely: labelled data.
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity.
LSD-C: Linearly Separable Deep Clusters
We present LSD-C, a novel method to identify clusters in an unlabeled dataset.
VGGSound: A Large-scale Audio-Visual Dataset
Our goal is to collect a large-scale audio-visual dataset with low label noise from videos in the wild using computer vision techniques.
Monocular Depth Estimation with Self-supervised Instance Adaptation
Recent advances in self-supervised learning havedemonstrated that it is possible to learn accurate monoculardepth reconstruction from raw video data, without using any 3Dground truth for supervision.
Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation
Remarkably, the resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks such as 3DPW.
Ranked #23 on
3D Human Pose Estimation
on MPI-INF-3DHP
(PA-MPJPE metric)
There and Back Again: Revisiting Backpropagation Saliency Methods
Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample.
Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives
Visuomotor control (VMC) is an effective means of achieving basic manipulation tasks such as pushing or pick-and-place from raw images.
Fixing the train-test resolution discrepancy: FixEfficientNet
An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88. 5% top-1 accuracy (top-5: 98. 7%), which establishes the new state of the art for ImageNet with a single crop.
Ranked #9 on
Image Classification
on ImageNet ReaL
(using extra training data)
On Compositions of Transformations in Contrastive Self-Supervised Learning
In the image domain, excellent representations can be learned by inducing invariance to content-preserving transformations via noise contrastive learning.
Transferring Dense Pose to Proximal Animal Classes
Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail.
Automatically Discovering and Learning New Visual Categories with Ranking Statistics
In this work we address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labeled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use rank statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data.
Correlated Uncertainty for Learning Dense Correspondences from Noisy Labels
We show that these models, by understanding uncertainty better, can solve the original DensePose task more accurately, thus setting the new state-of-the-art accuracy in this benchmark.
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision.
Self-labelling via simultaneous clustering and representation learning
Combining clustering and representation learning is one of the most promising approaches for unsupervised learning of deep neural networks.
Ranked #8 on
Contrastive Learning
on imagenet-1k
Occlusions for Effective Data Augmentation in Image Classification
Deep networks for visual recognition are known to leverage "easy to recognise" portions of objects such as faces and distinctive texture patterns.
NormGrad: Finding the Pixels that Matter for Training
In this paper, we are rather interested by the locations of an image that contribute to the model's training.
Understanding Deep Networks via Extremal Perturbations and Smooth Masks
In this paper, we discuss some of the shortcomings of existing approaches to perturbation analysis and address them by introducing the concept of extremal perturbations, which are theoretically grounded and interpretable.
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
We propose C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images.
Learning to Discover Novel Visual Categories via Deep Transfer Clustering
The second contribution is a method to estimate the number of classes in the unlabelled data.
Unsupervised Learning of Landmarks by Descriptor Vector Exchange
Equivariance to random image transformations is an effective method to learn landmarks of object categories, such as the eyes and the nose in faces, without manual supervision.
Ranked #1 on
Unsupervised Facial Landmark Detection
on 300W
AutoCorrect: Deep Inductive Alignment of Noisy Geometric Annotations
We propose AutoCorrect, a method to automatically learn object-annotation alignments from a dataset with annotations affected by geometric noise.
Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
We propose KeypointGAN, a new method for recognizing the pose of objects from a single image that for learning uses only unlabelled videos and a weak empirical prior on the object poses.
Fixing the train-test resolution discrepancy
Conversely, when training a ResNeXt-101 32x48d pre-trained in weakly-supervised fashion on 940 million public images at resolution 224x224 and further optimizing for test resolution 320x320, we obtain a test top-1 accuracy of 86. 4% (top-5: 98. 0%) (single-crop).
Ranked #2 on
Fine-Grained Image Classification
on Birdsnap
(using extra training data)
Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues
DensePose supersedes traditional landmark detectors by densely mapping image pixels to body surface coordinates.
Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images
Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder.
Unsupervised Intuitive Physics from Past Experiences
We are interested in learning models of intuitive physics similar to the ones that animals use for navigation, manipulation and planning.
Semi-Supervised Learning with Scarce Annotations
The first is a simple but effective one: we leverage the power of transfer learning among different tasks and self-supervision to initialize a good representation of the data without making use of any label.
Guiding Physical Intuition with Neural Stethoscopes
Using an adversarial stethoscope, the network is successfully de-biased, leading to a performance increase from 66% to 88%.
A critical analysis of self-supervision, or what we can learn from a single image
We look critically at popular self-supervision techniques for learning deep convolutional neural networks without manual labels.
MultiGrain: a unified image embedding for classes and instances
When fed to a linear classifier, the learned embeddings provide state-of-the-art classification accuracy.
Ranked #1 on
Image Retrieval
on INRIA Holidays
Modelling and unsupervised learning of symmetric deformable object categories
We propose a new approach to model and learn, without manual supervision, the symmetries of natural objects, such as faces or flowers, given only images as input.
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.
Learning to Read by Spelling: Towards Unsupervised Text Recognition
This work presents a method for visual text recognition without using any paired supervisory data.
Supervising the new with the old: learning SFM from SFM
First, since such self-supervised approaches are based on the brightness constancy assumption, which is valid only for a subset of pixels, we propose a probabilistic learning formulation where the network predicts distributions over variables rather than specific values.
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild
We make the following contributions: (i) we develop a strong teacher network for facial emotion recognition that achieves the state of the art on a standard benchmark; (ii) we use the teacher to train a student, tabula rasa, to learn representations (embeddings) for speech emotion recognition without access to labelled audio data; and (iii) we show that the speech emotion embedding can be used for speech emotion recognition on external benchmark datasets.
Ranked #3 on
Facial Expression Recognition (FER)
on FERPlus
Facial Emotion Recognition
Facial Expression Recognition (FER)
+1
Semi-convolutional Operators for Instance Segmentation
Object detection and instance segmentation are dominated by region-based methods such as Mask RCNN.
Inductive Visual Localisation: Factorised Training for Superior Generalisation
End-to-end trained Recurrent Neural Networks (RNNs) have been successfully applied to numerous problems that require processing sequences, such as image captioning, machine translation, and text recognition.
Large scale evaluation of local image feature detectors on homography datasets
The new protocol is better for assessment on a large number of images and reduces the dependency of the results on unwanted distractors such as the number of detected features and the feature magnification factor.
Invariant Information Clustering for Unsupervised Image Classification and Segmentation
The method is not specialised to computer vision and operates on any paired dataset samples; in our experiments we use random transforms to obtain a pair from each image.
Ranked #1 on
Unsupervised MNIST
on MNIST
Cross Pixel Optical Flow Similarity for Self-Supervised Learning
We propose a novel method for learning convolutional neural image representations without manual supervision.
Unsupervised Learning of Object Landmarks through Conditional Image Generation
We propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision.
Scrutinizing and De-Biasing Intuitive Physics with Neural Stethoscopes
Conversely, training on an easy dataset where visual cues are positively correlated with stability, the baseline model learns a bias leading to poor performance on a harder dataset.
MapNet: An Allocentric Spatial Memory for Mapping Environments
The module contains an allocentric spatial memory that can be accessed associatively by feeding to it the current sensory input, resulting in localization, and then updated using an LSTM or similar mechanism.
Meta-learning with differentiable closed-form solvers
The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data.
Small steps and giant leaps: Minimal Newton solvers for Deep Learning
Instead, we propose to keep a single estimate of the gradient projected by the inverse Hessian matrix, and update it once per iteration.
PyTorch CurveBall - A second-order optimizer for deep networks
We propose a fast second-order method that can be used as a drop-in replacementfor current deep learning solvers.
Unsupervised Intuitive Physics from Visual Observations
While learning models of intuitive physics is an increasingly active area of research, current approaches still fall short of natural intelligences in one important regard: they require external supervision, such as explicit access to physical states, at training and sometimes even at test times.
ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
Physical intuition is pivotal for intelligent agents to perform complex tasks.
Self-supervised Learning of Geometrically Stable Features Through Probabilistic Introspection
Self-supervision can dramatically cut back the amount of manually-labelled data required to train deep neural networks.
Efficient parametrization of multi-domain deep neural networks
A practical limitation of deep neural networks is their high degree of specialization to a single task and visual domain.
Long-term Tracking in the Wild: A Benchmark
We introduce the OxUvA dataset and benchmark for evaluating single-object tracking algorithms.
Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks
By studying such embeddings, we are able to show that 1., in most cases, multiple filters are required to code for a concept, that 2., often filters are not concept specific and help encode multiple concepts, and that 3., compared to single filter activations, filter embeddings are able to better characterize the meaning of a representation and its relationship to other concepts.
Taking Visual Motion Prediction To New Heightfields
In order to be able to leverage the approximation capabilities of artificial intelligence techniques in such physics related contexts, researchers have handcrafted the relevant states, and then used neural networks to learn the state transitions using simulation runs as training data.
Deep Image Prior
In this paper, we show that, on the contrary, the structure of a generator network is sufficient to capture a great deal of low-level image statistics prior to any learning.
Ranked #4 on
Feature Upsampling
on ImageNet
DeepRadiologyNet: Radiologist Level Pathology Detection in CT Head Images
Thus, DeepRadiologyNet enables significant reduction in the workload of human radiologists by automatically filtering studies and reporting on the high-confidence ones at an operating point well below the literal error rate for US Board Certified radiologists, estimated at 0. 82%.
Unsupervised learning of object frames by dense equivariant image labelling
One of the key challenges of visual perception is to extract abstract models of 3D objects and object categories from visual measurements, which are affected by complex nuisance factors such as viewpoint, occlusion, motion, and deformations.
Ranked #3 on
Unsupervised Facial Landmark Detection
on AFLW-MTFL
Learning to Represent Mechanics via Long-term Extrapolation and Interpolation
While the basic laws of Newtonian mechanics are well understood, explaining a physical scenario still requires manually modeling the problem with suitable equations and associated parameters.
Learning multiple visual domains with residual adapters
There is a growing interest in learning data representations that work well for many different types of problems and data.
Learning 3D Object Categories by Looking Around Them
Traditional approaches for learning 3D object categories use either synthetic data or manual supervision.
Unsupervised learning of object landmarks by factorized spatial embeddings
Learning automatically the structure of object categories remains an important open problem in computer vision.
Ranked #2 on
Unsupervised Facial Landmark Detection
on AFLW-MTFL
End-to-end representation learning for Correlation Filter based tracking
The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations.
Ranked #3 on
Visual Object Tracking
on OTB-50
HPatches: A benchmark and evaluation of handcrafted and learned local descriptors
In this paper, we propose a novel benchmark for evaluating local image descriptors.
AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching
Despite significant progress of deep learning in recent years, state-of-the-art semantic matching methods still rely on legacy features such as SIFT or HoG.
Interpretable Explanations of Black Boxes by Meaningful Perturbation
As machine learning algorithms are increasingly applied to high impact yet high risk tasks, such as medical diagnosis or autonomous driving, it is critical that researchers can explain how such algorithms arrived at their predictions.
It Takes (Only) Two: Adversarial Generator-Encoder Networks
Unlike previous hybrids of autoencoders and adversarial networks, the adversarial game in our approach is set up directly between the encoder and the generator, and no external mappings are trained in the process of learning.
Learning A Physical Long-term Predictor
Evolution has resulted in highly developed abilities in many natural intelligences to quickly and accurately predict mechanical phenomena.
Universal representations:The missing link between faces, text, planktons, and cat breeds
With the advent of large labelled datasets and high-capacity models, the performance of machine vision systems has been improving rapidly.
Ranked #14 on
Continual Learning
on visual domain decathlon (10 tasks)
Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis
The recent work of Gatys et al., who characterized the style of an image by the statistics of convolutional neural network filters, ignited a renewed interest in the texture generation and image stylization problems.
Action Recognition with Dynamic Image Networks
This is a powerful idea because it allows to convert any video to an image so that existing CNN models pre-trained for the analysis of still images can be immediately extended to videos.
Learning Grimaces by Watching TV
As a starting point, we consider the problem of relating facial expressions to objectively measurable events occurring in videos.
Warped Convolutions: Efficient Invariance to Spatial Transformations
Convolutional Neural Networks (CNNs) are extremely efficient, since they exploit the inherent translation-invariance of natural images.
Fully-Trainable Deep Matching
Deep Matching (DM) is a popular high-quality method for quasi-dense image matching.
Instance Normalization: The Missing Ingredient for Fast Stylization
It this paper we revisit the fast stylization method introduced in Ulyanov et.
Learning the semantic structure of objects from Web supervision
While recent research in image understanding has often focused on recognizing more types of objects, understanding more about the objects is just as important.
Fully-Convolutional Siamese Networks for Object Tracking
The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object's appearance exclusively online, using as sole training data the video itself.
Ranked #3 on
Visual Object Tracking
on OTB-50