Search Results for author:
Found 172 papers, 77 papers with code
HoloFusion: Towards Photo-realistic 3D Generative Modeling
Diffusion-based image generators can now produce high-quality and diverse samples, but their success has yet to fully translate to 3D generation: existing diffusion methods can either generate low-resolution but 3D consistent outputs, or detailed 2D views of 3D objects but with potential structural defects and lacking view consistency or realism.
Online Clustered Codebook
Vector Quantisation (VQ) is experiencing a comeback in machine learning, where it is increasingly used in representation learning.
Replay: Multi-modal Multi-view Acted Videos for Casual Holography
We introduce Replay, a collection of multi-view, multi-modal videos of humans interacting socially.
CoTracker: It is Better to Track Together
In this paper, we thus propose CoTracker, an architecture that jointly tracks multiple points throughout an entire video.
Diffusion Models for Zero-Shot Open-Vocabulary Segmentation
This provides a distribution of appearances for a given text circumventing the ambiguity problem.
Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data
We fit a diffusion model to a large number of viewsets for a given category of objects.
DynamicStereo: Consistent Dynamic Depth from Stereo Videos
The network learns to pool information from neighboring frames to improve the temporal consistency of its predictions.
Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion
We propose a framework using an image generator like Stable Diffusion to generate virtual training data for learning such a reconstruction network from scratch.
What does CLIP know about a red circle? Visual prompt engineering for VLMs
Large-scale Vision-Language Models, such as CLIP, learn powerful image-text representations that have found numerous applications, from zero-shot classification to text-to-image generation.
Continual Detection Transformer for Incremental Object Detection
Incremental object detection (IOD) aims to train an object detector in phases, each with annotations for new object categories.
Training-Free Layout Control with Cross-Attention Guidance
We thoroughly evaluate our approach on three benchmarks and provide several qualitative examples and a comparative analysis of the two strategies that demonstrate the superiority of backward guidance compared to forward guidance, as well as prior work.
HoloDiffusion: Training a 3D Diffusion Model using 2D Images
We show that our diffusion models are scalable, train robustly, and are competitive in terms of sample quality and fidelity to existing approaches for 3D generative modeling.
Real-time volumetric rendering of dynamic humans
We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits.
RealFusion: 360° Reconstruction of Any Object from a Single Image
We consider the problem of reconstructing a full 360{\deg} photographic model of an object from a single image of it.
$PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction
Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision.
Text-To-4D Dynamic Scene Generation
We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions.
Novel-View Acoustic Synthesis
We introduce the novel-view acoustic synthesis (NVAS) task: given the sight and sound observed at a source viewpoint, can we synthesize the sound of that scene from an unseen target viewpoint?
PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction
Reconstructing the 3D shape of an object from a single RGB image is a long-standing problem in computer vision.
RealFusion: 360deg Reconstruction of Any Object From a Single Image
We consider the problem of reconstructing a full 360deg photographic model of an object from a single image of it.
Self-Supervised Correspondence Estimation via Multiview Registration
To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences.
MagicPony: Learning Articulated 3D Animals in the Wild
We consider the problem of predicting the 3D shape, articulation, viewpoint, texture, and lighting of an articulated animal like a horse given a single test image as input.
Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories
Obtaining photorealistic reconstructions of objects from sparse views is inherently ambiguous and can only be achieved by learning suitable reconstruction priors.
Unsupervised Multi-object Segmentation by Predicting Probable Motion Patterns
We propose a new approach to learn to segment multiple image objects without manual supervision.
Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing
Self-supervised visual representation learning has recently attracted significant research interest.
Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations
We present Neural Feature Fusion Fields (N3F), a method that improves dense 2D image feature extractors when the latter are applied to the analysis of multiple images reconstructible as a 3D scene.
SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data
We present a method for the accurate 3D reconstruction of partly-symmetric objects.
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization
We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene.
Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion
Motion, measured via optical flow, provides a powerful cue to discover and learn objects in images and videos.
Ranked #4 on
Unsupervised Object Segmentation
on SegTrack-v2
End-to-End Visual Editing with a Generatively Pre-Trained Artist
We consider the targeted image editing problem: blending a region in a source image with a driver image that specifies the desired change.
Generalized Category Discovery
Here, the unlabelled images may come from labelled classes or from novel ones.
Ranked #1 on
Open-World Semi-Supervised Learning
on CIFAR-10
(Seen accuracy (50% Labeled) metric)
Fine-Grained Visual Recognition
Open-World Semi-Supervised Learning
+1
KeyTr: Keypoint Transporter for 3D Reconstruction of Deformable Objects in Videos
Compared to weaker deformation models, this significantly reduces the reconstruction ambiguity and, for dynamic objects, allows Keypoint Transporter to obtain reconstructions of the quality superior or at least comparable to prior approaches while being much faster and reliant on a pre-trained monocular depth estimator network.
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model.
Audio-Visual Synchronisation in the wild
Finally, we set the first benchmark for general audio-visual synchronisation with over 160 diverse classes in the new VGG-Sound Sync video dataset.
Unsupervised Part Discovery from Contrastive Reconstruction
First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts.
Ranked #1 on
Unsupervised Keypoint Estimation
on CUB
The Curious Layperson: Fine-Grained Image Recognition without Expert Labels
We then train a fine-grained textual similarity model that matches image descriptions with documents on a sentence-level basis.
NeuralDiff: Segmenting 3D objects that move in egocentric videos
Given a raw video sequence taken from a freely-moving camera, we study the problem of decomposing the observed 3D scene into a static background and a dynamic foreground containing the objects that move in the video sequence.
Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
In this paper, we first demonstrate that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes.
Ranked #10 on
Out-of-Distribution Detection
on CIFAR-100 vs CIFAR-10
Measuring the Interpretability of Unsupervised Representations via Quantized Reversed Probing
Self-supervised visual representation learning has attracted significant research interest.
Ranked #89 on
Image Classification
on ObjectNet
(using extra training data)
PASS: An ImageNet replacement for self-supervised pretraining without humans
On the other hand, state-of-the-art pretraining is nowadays obtained with unsupervised methods, meaning that labelled datasets such as ImageNet may not be necessary, or perhaps not even optimal, for model pretraining.
Real Time Monocular Vehicle Velocity Estimation using Synthetic Data
Vision is one of the primary sensing modalities in autonomous driving.
Lifting 2D Object Locations to 3D by Discounting LiDAR Outliers across Objects and Views
We present a system for automatic converting of 2D mask object predictions and raw LiDAR point clouds into full 3D bounding boxes of objects.
DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension
The method learns, in an end-to-end fashion, a soft partition of a given category-specific 3D template mesh into rigid parts together with a monocular reconstruction network that predicts the part motions such that they reproject correctly onto 2D DensePose-like surface annotations of the object.
Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields
Implicit neural representation is a recent approach to learn shape collections as zero level-sets of neural networks, where each shape is represented by a latent code.
DOVE: Learning Deformable 3D Objects by Watching Videos
In this paper, we present DOVE, a method that learns textured 3D models of deformable object categories from monocular videos available online, without keypoint, viewpoint or template shape supervision.
AutoNovel: Automatically Discovering and Learning Novel Visual Categories
We present a new approach called AutoNovel to address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labelled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use ranking statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data.
Ranked #1 on
Novel Class Discovery
on SVHN
Pedestrian and Ego-Vehicle Trajectory Prediction From Monocular Camera
Predicting future pedestrian trajectory is a crucial component of autonomous driving systems, as recognizing critical situations based only on current pedestrian position may come too late for any meaningful corrective action (e. g. breaking) to take place.
NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go
We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes and produces in one go, i. e. in a single feed forward pass, a smooth interpolation and point-to-point correspondences between them.
Discovering Relationships between Object Categories via Universal Canonical Maps
Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects.
Test Sample Accuracy Scales with Training Sample Density in Neural Networks
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in representation space.
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers
In video transformers, the time dimension is often treated in the same way as the two spatial dimensions.
Ranked #14 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
Recent research has shown that numerous human-interpretable directions exist in the latent space of GANs.
Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes
We build on the idea of view synthesis, which uses classical camera geometry to re-render a source image from a different point-of-view, specified by a predicted relative pose and depth map.
Self-supervised object detection from audio-visual correspondence
We tackle the problem of learning object detectors without supervision.
Localizing Visual Sounds the Hard Way
We show that our algorithm achieves state-of-the-art performance on the popular Flickr SoundNet dataset.
Unsupervised Learning of 3D Object Categories from Videos in the Wild
Our goal is to learn a deep network that, given a small number of images of an object of a given category, reconstructs it in 3D.
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning
First, for space, we show that spatial augmentations such as cropping do work well for videos too, but that previous implementations, due to the high processing and memory cost, could not do this at a scale sufficient for it to work well.
Continuous Surface Embeddings
In this work, we focus on the task of learning and representing dense correspondences in deformable object categories.
3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data
We consider the problem of obtaining dense 3D reconstructions of humans from single and partially occluded views.
Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
The increasing impact of black box models, and particularly of unsupervised ones, comes with an increasing interest in tools to understand and interpret them.
Support-set bottlenecks for video-text representation learning
The dominant paradigm for learning video-text representations -- noise contrastive learning -- increases the similarity of the representations of pairs of samples that are known to be related, such as text and video from the same sample, and pushes away the representations of all other pairs.
Multi-modal Self-Supervision from Generalized Data Transformations
In this paper, we show that, for videos, the answer is more complex, and that better results can be obtained by accounting for the interplay between invariance, distinctiveness, multiple modalities and time.
Calibrating Self-supervised Monocular Depth Estimation
In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal.
Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstruction
We propose the Canonical 3D Deformer Map, a new representation of the 3D shape of common object categories that can be learned from a collection of 2D images of independent objects.
RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces
We present RELATE, a model that learns to generate physically plausible scenes and videos of multiple interacting objects.
Labelling unlabelled videos from scratch with multi-modal self-supervision
A large part of the current success of deep learning lies in the effectiveness of data -- more precisely: labelled data.
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity.
LSD-C: Linearly Separable Deep Clusters
We present LSD-C, a novel method to identify clusters in an unlabeled dataset.
VGGSound: A Large-scale Audio-Visual Dataset
Our goal is to collect a large-scale audio-visual dataset with low label noise from videos in the wild using computer vision techniques.
Monocular Depth Estimation with Self-supervised Instance Adaptation
Recent advances in self-supervised learning havedemonstrated that it is possible to learn accurate monoculardepth reconstruction from raw video data, without using any 3Dground truth for supervision.
Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation
Remarkably, the resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks such as 3DPW.
Ranked #20 on
3D Human Pose Estimation
on MPI-INF-3DHP
(PA-MPJPE metric)
There and Back Again: Revisiting Backpropagation Saliency Methods
Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample.
Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives
Visuomotor control (VMC) is an effective means of achieving basic manipulation tasks such as pushing or pick-and-place from raw images.
Fixing the train-test resolution discrepancy: FixEfficientNet
An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88. 5% top-1 accuracy (top-5: 98. 7%), which establishes the new state of the art for ImageNet with a single crop.
Ranked #8 on
Image Classification
on ImageNet ReaL
(using extra training data)
On Compositions of Transformations in Contrastive Self-Supervised Learning
In the image domain, excellent representations can be learned by inducing invariance to content-preserving transformations via noise contrastive learning.
Transferring Dense Pose to Proximal Animal Classes
Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail.
Automatically Discovering and Learning New Visual Categories with Ranking Statistics
In this work we address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labeled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use rank statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data.
Correlated Uncertainty for Learning Dense Correspondences from Noisy Labels
We show that these models, by understanding uncertainty better, can solve the original DensePose task more accurately, thus setting the new state-of-the-art accuracy in this benchmark.
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision.
Self-labelling via simultaneous clustering and representation learning
Combining clustering and representation learning is one of the most promising approaches for unsupervised learning of deep neural networks.
Ranked #6 on
Image Clustering
on ImageNet
Occlusions for Effective Data Augmentation in Image Classification
Deep networks for visual recognition are known to leverage "easy to recognise" portions of objects such as faces and distinctive texture patterns.
NormGrad: Finding the Pixels that Matter for Training
In this paper, we are rather interested by the locations of an image that contribute to the model's training.
Understanding Deep Networks via Extremal Perturbations and Smooth Masks
In this paper, we discuss some of the shortcomings of existing approaches to perturbation analysis and address them by introducing the concept of extremal perturbations, which are theoretically grounded and interpretable.
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
We propose C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images.
Learning to Discover Novel Visual Categories via Deep Transfer Clustering
The second contribution is a method to estimate the number of classes in the unlabelled data.
Unsupervised Learning of Landmarks by Descriptor Vector Exchange
Equivariance to random image transformations is an effective method to learn landmarks of object categories, such as the eyes and the nose in faces, without manual supervision.
Ranked #1 on
Unsupervised Facial Landmark Detection
on 300W
AutoCorrect: Deep Inductive Alignment of Noisy Geometric Annotations
We propose AutoCorrect, a method to automatically learn object-annotation alignments from a dataset with annotations affected by geometric noise.
Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
We propose KeypointGAN, a new method for recognizing the pose of objects from a single image that for learning uses only unlabelled videos and a weak empirical prior on the object poses.
Fixing the train-test resolution discrepancy
Conversely, when training a ResNeXt-101 32x48d pre-trained in weakly-supervised fashion on 940 million public images at resolution 224x224 and further optimizing for test resolution 320x320, we obtain a test top-1 accuracy of 86. 4% (top-5: 98. 0%) (single-crop).
Ranked #2 on
Fine-Grained Image Classification
on Birdsnap
(using extra training data)
Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues
DensePose supersedes traditional landmark detectors by densely mapping image pixels to body surface coordinates.
Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images
Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder.
Unsupervised Intuitive Physics from Past Experiences
We are interested in learning models of intuitive physics similar to the ones that animals use for navigation, manipulation and planning.
Semi-Supervised Learning with Scarce Annotations
The first is a simple but effective one: we leverage the power of transfer learning among different tasks and self-supervision to initialize a good representation of the data without making use of any label.
Guiding Physical Intuition with Neural Stethoscopes
Using an adversarial stethoscope, the network is successfully de-biased, leading to a performance increase from 66% to 88%.
A critical analysis of self-supervision, or what we can learn from a single image
We look critically at popular self-supervision techniques for learning deep convolutional neural networks without manual labels.
MultiGrain: a unified image embedding for classes and instances
When fed to a linear classifier, the learned embeddings provide state-of-the-art classification accuracy.
Ranked #1 on
Image Retrieval
on INRIA Holidays
Modelling and unsupervised learning of symmetric deformable object categories
We propose a new approach to model and learn, without manual supervision, the symmetries of natural objects, such as faces or flowers, given only images as input.
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.
Learning to Read by Spelling: Towards Unsupervised Text Recognition
This work presents a method for visual text recognition without using any paired supervisory data.
Supervising the new with the old: learning SFM from SFM
First, since such self-supervised approaches are based on the brightness constancy assumption, which is valid only for a subset of pixels, we propose a probabilistic learning formulation where the network predicts distributions over variables rather than specific values.
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild
We make the following contributions: (i) we develop a strong teacher network for facial emotion recognition that achieves the state of the art on a standard benchmark; (ii) we use the teacher to train a student, tabula rasa, to learn representations (embeddings) for speech emotion recognition without access to labelled audio data; and (iii) we show that the speech emotion embedding can be used for speech emotion recognition on external benchmark datasets.
Ranked #3 on
Facial Expression Recognition (FER)
on FERPlus
Facial Emotion Recognition
Facial Expression Recognition (FER)
+1
Semi-convolutional Operators for Instance Segmentation
Object detection and instance segmentation are dominated by region-based methods such as Mask RCNN.
Inductive Visual Localisation: Factorised Training for Superior Generalisation
End-to-end trained Recurrent Neural Networks (RNNs) have been successfully applied to numerous problems that require processing sequences, such as image captioning, machine translation, and text recognition.
Large scale evaluation of local image feature detectors on homography datasets
The new protocol is better for assessment on a large number of images and reduces the dependency of the results on unwanted distractors such as the number of detected features and the feature magnification factor.
Invariant Information Clustering for Unsupervised Image Classification and Segmentation
The method is not specialised to computer vision and operates on any paired dataset samples; in our experiments we use random transforms to obtain a pair from each image.
Ranked #1 on
Unsupervised MNIST
on MNIST
Cross Pixel Optical Flow Similarity for Self-Supervised Learning
We propose a novel method for learning convolutional neural image representations without manual supervision.
Unsupervised Learning of Object Landmarks through Conditional Image Generation
We propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision.
Conditional Image Generation
Unsupervised Facial Landmark Detection
Scrutinizing and De-Biasing Intuitive Physics with Neural Stethoscopes
Conversely, training on an easy dataset where visual cues are positively correlated with stability, the baseline model learns a bias leading to poor performance on a harder dataset.
MapNet: An Allocentric Spatial Memory for Mapping Environments
The module contains an allocentric spatial memory that can be accessed associatively by feeding to it the current sensory input, resulting in localization, and then updated using an LSTM or similar mechanism.
PyTorch CurveBall - A second-order optimizer for deep networks
We propose a fast second-order method that can be used as a drop-in replacementfor current deep learning solvers.
Small steps and giant leaps: Minimal Newton solvers for Deep Learning
Instead, we propose to keep a single estimate of the gradient projected by the inverse Hessian matrix, and update it once per iteration.
Meta-learning with differentiable closed-form solvers
The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data.
Unsupervised Intuitive Physics from Visual Observations
While learning models of intuitive physics is an increasingly active area of research, current approaches still fall short of natural intelligences in one important regard: they require external supervision, such as explicit access to physical states, at training and sometimes even at test times.
ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
Physical intuition is pivotal for intelligent agents to perform complex tasks.
Self-supervised Learning of Geometrically Stable Features Through Probabilistic Introspection
Self-supervision can dramatically cut back the amount of manually-labelled data required to train deep neural networks.
Efficient parametrization of multi-domain deep neural networks
A practical limitation of deep neural networks is their high degree of specialization to a single task and visual domain.
Long-term Tracking in the Wild: A Benchmark
We introduce the OxUvA dataset and benchmark for evaluating single-object tracking algorithms.
Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks
By studying such embeddings, we are able to show that 1., in most cases, multiple filters are required to code for a concept, that 2., often filters are not concept specific and help encode multiple concepts, and that 3., compared to single filter activations, filter embeddings are able to better characterize the meaning of a representation and its relationship to other concepts.
Taking Visual Motion Prediction To New Heightfields
In order to be able to leverage the approximation capabilities of artificial intelligence techniques in such physics related contexts, researchers have handcrafted the relevant states, and then used neural networks to learn the state transitions using simulation runs as training data.
Deep Image Prior
In this paper, we show that, on the contrary, the structure of a generator network is sufficient to capture a great deal of low-level image statistics prior to any learning.
DeepRadiologyNet: Radiologist Level Pathology Detection in CT Head Images
Thus, DeepRadiologyNet enables significant reduction in the workload of human radiologists by automatically filtering studies and reporting on the high-confidence ones at an operating point well below the literal error rate for US Board Certified radiologists, estimated at 0. 82%.
Unsupervised learning of object frames by dense equivariant image labelling
One of the key challenges of visual perception is to extract abstract models of 3D objects and object categories from visual measurements, which are affected by complex nuisance factors such as viewpoint, occlusion, motion, and deformations.
Ranked #3 on
Unsupervised Facial Landmark Detection
on AFLW-MTFL
Optical Flow Estimation
Unsupervised Facial Landmark Detection
Learning to Represent Mechanics via Long-term Extrapolation and Interpolation
While the basic laws of Newtonian mechanics are well understood, explaining a physical scenario still requires manually modeling the problem with suitable equations and associated parameters.
Learning multiple visual domains with residual adapters
There is a growing interest in learning data representations that work well for many different types of problems and data.
Learning 3D Object Categories by Looking Around Them
Traditional approaches for learning 3D object categories use either synthetic data or manual supervision.
Unsupervised learning of object landmarks by factorized spatial embeddings
Learning automatically the structure of object categories remains an important open problem in computer vision.
Ranked #2 on
Unsupervised Facial Landmark Detection
on AFLW-MTFL
Unsupervised Facial Landmark Detection
Unsupervised Human Pose Estimation
+1
End-to-end representation learning for Correlation Filter based tracking
The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations.
Ranked #3 on
Visual Object Tracking
on OTB-50
HPatches: A benchmark and evaluation of handcrafted and learned local descriptors
In this paper, we propose a novel benchmark for evaluating local image descriptors.
AnchorNet: A Weakly Supervised Network to Learn Geometry-sensitive Features For Semantic Matching
Despite significant progress of deep learning in recent years, state-of-the-art semantic matching methods still rely on legacy features such as SIFT or HoG.
Interpretable Explanations of Black Boxes by Meaningful Perturbation
As machine learning algorithms are increasingly applied to high impact yet high risk tasks, such as medical diagnosis or autonomous driving, it is critical that researchers can explain how such algorithms arrived at their predictions.
It Takes (Only) Two: Adversarial Generator-Encoder Networks
Unlike previous hybrids of autoencoders and adversarial networks, the adversarial game in our approach is set up directly between the encoder and the generator, and no external mappings are trained in the process of learning.
Learning A Physical Long-term Predictor
Evolution has resulted in highly developed abilities in many natural intelligences to quickly and accurately predict mechanical phenomena.
Universal representations:The missing link between faces, text, planktons, and cat breeds
With the advent of large labelled datasets and high-capacity models, the performance of machine vision systems has been improving rapidly.
Ranked #14 on
Continual Learning
on visual domain decathlon (10 tasks)
Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis
The recent work of Gatys et al., who characterized the style of an image by the statistics of convolutional neural network filters, ignited a renewed interest in the texture generation and image stylization problems.
Action Recognition with Dynamic Image Networks
This is a powerful idea because it allows to convert any video to an image so that existing CNN models pre-trained for the analysis of still images can be immediately extended to videos.
Learning Grimaces by Watching TV
As a starting point, we consider the problem of relating facial expressions to objectively measurable events occurring in videos.
Warped Convolutions: Efficient Invariance to Spatial Transformations
Convolutional Neural Networks (CNNs) are extremely efficient, since they exploit the inherent translation-invariance of natural images.
Fully-Trainable Deep Matching
Deep Matching (DM) is a popular high-quality method for quasi-dense image matching.
Instance Normalization: The Missing Ingredient for Fast Stylization
It this paper we revisit the fast stylization method introduced in Ulyanov et.
Learning the semantic structure of objects from Web supervision
While recent research in image understanding has often focused on recognizing more types of objects, understanding more about the objects is just as important.
Fully-Convolutional Siamese Networks for Object Tracking
The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object's appearance exclusively online, using as sole training data the video itself.
Ranked #3 on
Visual Object Tracking
on OTB-50
Learning feed-forward one-shot learners
In this paper, we propose a method to learn the parameters of a deep model in one shot.
Integrated perception with recurrent multi-task neural networks
Modern discriminative predictors have been shown to match natural intelligences in specific perceptual tasks in image classification, object and part detection, boundary extraction, etc.
Dynamic Image Networks for Action Recognition
We introduce the concept of dynamic image, a novel compact representation of videos useful for video analysis especially when convolutional neural networks (CNNs) are used.
Ranked #61 on
Action Recognition
on HMDB-51
Learning Covariant Feature Detectors
We support these ideas theoretically, proposing a novel analysis of local features in term of geometric transformations, and we show that all common and many uncommon detectors can be derived in this framework.
Synthetic Data for Text Localisation in Natural Images
In this paper we introduce a new method for text detection in natural images.
Ranked #12 on
Scene Text Detection
on ICDAR 2013 (1015)
Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
Gatys et al. recently demonstrated that deep networks can generate beautiful textures and stylized images from a single texture example.
Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
Image representations, from SIFT and bag of visual words to Convolutional Neural Networks (CNNs) are a crucial component of almost all computer vision systems.
Weakly Supervised Deep Detection Networks
Weakly supervised learning of object detection is an important problem in image understanding that still does not have a satisfactory solution.
Ranked #3 on
Weakly Supervised Object Detection
on HICO-DET
Deep filter banks for texture recognition, description, and segmentation
Visual textures have played a key role in image understanding because they convey important semantics of images, and because texture representations that pool local image descriptors in an orderless manner have had a tremendous impact in diverse applications.
R-CNN minus R
In object detection, methods such as R-CNN have obtained excellent results by integrating CNNs with region proposal generation algorithms such as selective search.
Deep Filter Banks for Texture Recognition and Segmentation
Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications.
Understanding the Fisher Vector: a multimodal part model
Fisher Vectors and related orderless visual statistics have demonstrated excellent performance in object detection, sometimes superior to established approaches such as the Deformable Part Models.
Automatic Discovery and Optimization of Parts for Image Classification
First, a collection of informative parts is discovered, using heuristics that promote part distinctiveness and diversity, and then classifiers are trained on the vector of part responses.
Deep Structured Output Learning for Unconstrained Text Recognition
We develop a representation suitable for the unconstrained recognition of words in natural images: the general case of no fixed lexicon and unknown length.
MatConvNet - Convolutional Neural Networks for MATLAB
MatConvNet is an implementation of Convolutional Neural Networks (CNNs) for MATLAB.
Reading Text in the Wild with Convolutional Neural Networks
In this work we present an end-to-end system for text spotting -- localising and recognising text in natural scene images -- and text based image retrieval.
Ranked #15 on
Scene Text Detection
on ICDAR 2013 (1015)
Understanding Deep Image Representations by Inverting Them
Image representations, from SIFT and Bag of Visual Words to Convolutional Neural Networks (CNNs), are a crucial component of almost any image understanding system.
Deep convolutional filter banks for texture recognition and segmentation
Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications.
Understanding image representations by measuring their equivariance and equivalence
Despite the importance of image representations such as histograms of oriented gradients and deep Convolutional Neural Networks (CNN), our theoretical understanding of them remains limited.
Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
In this work we present a framework for the recognition of natural scene text.
Ranked #32 on
Scene Text Recognition
on SVT
A Compact and Discriminative Face Track Descriptor
Our goal is to learn a compact, discriminative vector representation of a face track, suitable for the face recognition tasks of verification and classification.
Understanding Objects in Detail with Fine-Grained Attributes
We show that the collected data can be used to study the relation between part detection and attribute prediction by diagnosing the performance of classifiers that pool information from different parts of an object.
Speeding up Convolutional Neural Networks with Low Rank Expansions
The focus of this paper is speeding up the evaluation of convolutional neural networks.
Return of the Devil in the Details: Delving Deep into Convolutional Nets
In particular, we show that the data augmentation techniques commonly applied to CNN-based methods can also be applied to shallow methods, and result in an analogous performance boost.
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets).
Deep Fisher Networks for Large-Scale Image Classification
As massively parallel computations have become broadly available with modern GPUs, deep architectures trained on very large datasets have risen in popularity.
Describing Textures in the Wild
Patterns and textures are defining characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly.
Fine-Grained Visual Classification of Aircraft
This paper introduces FGVC-Aircraft, a new dataset containing 10, 000 images of aircraft spanning 100 aircraft models, organised in a three-level hierarchy.
Blocks That Shout: Distinctive Parts for Scene Classification
The automatic discovery of distinctive parts for an object or scene class is challenging since it requires simultaneously to learn the part appearance and also to identify the part occurrences in images.
Pylon Model for Semantic Segmentation
Often, the random field is applied over a flat partitioning of the image into non-intersecting elements, such as pixels or super-pixels.
Simultaneous Object Detection and Ranking with Weak Supervision
A standard approach to learning object category detectors is to provide strong supervision in the form of a region of interest (ROI) specifying each instance of the object in the training images.
Structured output regression for detection with partial truncation
We develop a structured output model for object category detection that explicitly accounts for alignment, multiple aspects and partial truncation in both training and inference.