Search Results for author:
Found 164 papers, 56 papers with code
VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion
To enable such capability in AI systems, we propose VoxFormer, a Transformer-based semantic scene completion framework that can output complete 3D volumetric semantics from only 2D images.
Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domain.
Synthesizing Physical Character-Scene Interactions
These scene interactions are learned using an adversarial discriminator that evaluates the realism of a motion within the context of a scene.
PADL: Language-Directed Physics-Based Character Control
In this work, we present PADL, which leverages recent innovations in NLP in order to take steps towards developing language-directed controllers for physics-based character animation.
Magic3D: High-Resolution Text-to-3D Content Creation
DreamFusion has recently demonstrated the utility of a pre-trained text-to-image diffusion model to optimize Neural Radiance Fields (NeRF), achieving remarkable text-to-3D synthesis results.
LION: Latent Point Diffusion Models for 3D Shape Generation
To advance 3D DDMs and make them useful for digital artists, we require (i) high generation quality, (ii) flexibility for manipulation and applications such as conditional synthesis and shape interpolation, and (iii) the ability to output smooth surfaces or meshes.
Ranked #1 on
Point Cloud Generation
on ShapeNet
XDGAN: Multi-Modal 3D Shape Generation in 2D Space
Specifically, we propose a novel method to convert 3D shapes into compact 1-channel geometry images and leverage StyleGAN3 and image-to-image translation networks to generate 3D objects in 2D space.
Optimizing Data Collection for Machine Learning
Modern deep learning systems require huge data sets to achieve impressive performance, but there is little guidance on how much or what kind of data to collect.
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations
VISOR annotates videos from EPIC-KITCHENS, which comes with a new set of challenges not encountered in current video segmentation datasets.
GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images
As several industries are moving towards modeling massive 3D virtual worlds, the need for content creation tools that can scale in terms of the quantity, quality, and diversity of 3D content is becoming evident.
Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion
In this work, we propose a neural approach that estimates the 5D HDR light field from a single image, and a differentiable object insertion formulation that enables end-to-end training with image-based losses that encourage realism.
MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentation
As a result, the learned 2D representations are view-invariant and geometrically consistent, leading to better generalization when trained on a limited number of labeled shapes compared to alternatives that utilize self-supervision in 2D or 3D alone.
Improving Semantic Segmentation in Transformers using Hierarchical Inter-Level Attention
HILA extends hierarchical vision transformer architectures by adding local connections between features of higher and lower levels to the backbone encoder.
How Much More Data Do I Need? Estimating Requirements for Downstream Tasks
Given a small training data set and a learning algorithm, how much more data is necessary to reach a target validation or test performance?
Scalable Neural Data Server: A Data Recommender for Transfer Learning
NDS uses a mixture of experts trained on data sources to estimate similarity between each source and the downstream task.
Variable Bitrate Neural Fields
Neural approximations of scalar and vector fields, such as signed distance functions and radiance fields, have emerged as accurate, high-quality representations.
Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps
Modern image generative models show remarkable sample quality when trained on a single domain or class of objects.
ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters
By leveraging a massively parallel GPU-based simulator, we are able to train skill embeddings using over a decade of simulated experiences, enabling our model to learn a rich and versatile repertoire of skills.
M$^2$BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation
In this paper, we propose M$^2$BEV, a unified framework that jointly performs 3D object detection and map segmentation in the Birds Eye View~(BEV) space with multi-camera image inputs.
AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis
In this paper, we address the problem of texture representation for 3D shapes for the challenging and underexplored tasks of texture transfer and synthesis.
Learning Smooth Neural Functions via Lipschitz Regularization
The latent descriptor of a neural field acts as a deformation handle for the 3D shape it represents.
Domain Adversarial Training: A Game Perspective
Defining optimal solutions in domain-adversarial training as a local Nash equilibrium, we show that gradient descent in domain-adversarial training can violate the asymptotic convergence guarantees of the optimizer, oftentimes hindering the transfer performance.
Causal Scene BERT: Improving object detection by searching for challenging groups of data
Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes.
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks
Standard Federated Learning (FL) techniques are limited to clients with identical network architectures.
BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations
By training an effective feature segmentation architecture on top of BigGAN, we turn BigGAN into a labeled dataset generator.
Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior
Scenario generation is formulated as an optimization in the latent space of this traffic model, perturbing an initial real-world scene to produce trajectories that collide with a given planner.
Frame Averaging for Equivariant Shape Space Learning
A natural way to incorporate symmetries in shape space learning is to ask that the mapping to the shape space (encoder) and mapping from the shape space (decoder) are equivariant to the relevant symmetries.
Hierarchical Neural Implicit Pose Network for Animation and Motion Retargeting
We present HIPNet, a neural implicit pose network trained on multiple subjects across many poses.
Don’t Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead.
Neural Fields as Learnable Kernels for 3D Reconstruction
We present Neural Kernel Fields: a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression.
Extracting Triangular 3D Models, Materials, and Lighting From Images
We present an efficient method for joint optimization of topology, materials and lighting from multi-view image observations.
Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation
Alternative solutions seek to exploit driving simulators that can generate large amounts of labeled data with a plethora of content variations.
Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis
The core of DMTet includes a deformable tetrahedral grid that encodes a discretized signed distance function and a differentiable marching tetrahedra layer that converts the implicit signed distance representation to the explicit surface mesh representation.
EditGAN: High-Precision Semantic Image Editing
EditGAN builds on a GAN framework that jointly models images and their semantic segmentations, requiring only a handful of labeled examples, making it a scalable tool for editing.
Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead.
DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer
We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiable renderers.
ATISS: Autoregressive Transformers for Indoor Scene Synthesis
The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many applications, from better interactive 3D tools to data synthesis for training and simulation.
Low-Budget Active Learning via Wasserstein Distance: An Integer Programming Approach
Active learning is the process of training a model with limited labeled data by selecting a core subset of an unlabeled data pool to label.
Causal Scene BERT: Improving object detection by searching for challenging groups
We verify that the prioritized groups found via intervention are challenging for the object detector and show that retraining with data collected from these groups helps inordinately compared to adding more IID data.
Physics-based Human Motion Estimation and Synthesis from Videos
By enabling learning of motion synthesis from video, our method paves the way for large-scale, realistic and diverse motion synthesis.
Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting
In this paper, we propose a unified, learning-based inverse rendering framework that formulates 3D spatially-varying lighting.
3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations
Given a small set of high-quality textured objects, our method can create many novel stylized shapes, resulting in effortless 3D content creation and style-ware data augmentation.
NP-DRAW: A Non-Parametric Structured Latent Variable Model for Image Generation
1) We propose a non-parametric prior distribution over the appearance of image parts so that the latent variable ``what-to-draw'' per step becomes a categorical random variable.
f-Domain-Adversarial Learning: Theory and Algorithms
Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain, and a related labeled dataset.
Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach
Active learning is the process of training a model with limited labeled data by selecting a core subset of an unlabeled data pool to label.
DriveGAN: Towards a Controllable High-Quality Neural Simulation
Realistic simulators are critical for training and verifying robotics systems.
Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets
This is expensive, and guaranteeing the quality of the labels is a major challenge.
DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort
To showcase the power of our approach, we generated datasets for 7 image segmentation tasks which include pixel-level labels for 34 human face parts, and 32 car parts.
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
Training deep networks with limited labeled data while achieving a strong generalization ability is key in the quest to reduce human annotation efforts.
Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level.
gradSim: Differentiable simulation for system identification and visuomotor control
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences.
Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks
The INN allows us to compute the inverse mapping of the homeomorphism, which in turn, enables the efficient computation of both the implicit surface function of a primitive and its mesh, without any additional post-processing.
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs, while achieving state-of-the-art geometry reconstruction quality.
f-Domain-Adversarial Learning: Theory and Algorithms for Unsupervised Domain Adaptation with Neural Networks
We provide empirical results for several f-divergences and show that some, not considered previously in domain-adversarial learning, achieve state-of-the-art results in practice.
Differentially Private Generative Models Through Optimal Transport
Generative models trained with privacy constraints on private data can sidestep this challenge and provide indirect access to the private data instead.
Personalized Federated Learning with First Order Model Optimization
While federated learning traditionally aims to train a single global model across decentralized local datasets, one model may not always be ideal for all participating clients.
Variational Amodal Object Completion
In images of complex scenes, objects are often occluding each other which makes perception tasks such as object detection and tracking, or robotic control tasks such as planning, challenging.
UniCon: Universal Neural Controller For Physics-based Character Motion
The field of physics-based animation is gaining importance due to the increasing demand for realism in video games and films, and has recently seen wide adoption of data-driven techniques, such as deep reinforcement learning (RL), which learn control from (human) demonstrations.
Emergent Road Rules In Multi-Agent Driving Environments
For autonomous vehicles to safely share the road with human drivers, autonomous vehicles must abide by specific "road rules" that human drivers have agreed to follow.
Learning Deformable Tetrahedral Meshes for 3D Reconstruction
We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
In this paper, we introduce Watch-And-Help (WAH), a challenge for testing social intelligence in agents.
The efficacy of Neural Planning Metrics: A meta-analysis of PKL on nuScenes
A high-performing object detection system plays a crucial role in autonomous driving (AD).
Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
Key to our approach is to exploit GANs as a multi-view data generator to train an inverse graphics network using an off-the-shelf differentiable renderer, and the trained inverse graphics network as a teacher to disentangle the GAN's latent code into interpretable 3D properties.
Fed-Sim: Federated Simulation for Medical Imaging
Since the model of geometry and material is disentangled from the imaging sensor, it can effectively be trained across multiple medical centers.
Expressive Telepresence via Modular Codec Avatars
It is important to note that traditional person-specific CAs are learned from few training samples, and typically lack robustness as well as limited expressiveness when transferring facial expressions.
Interactive Annotation of 3D Object Geometry using 2D Scribbles
We implement our framework as a web service and conduct a user study, where we show that user annotated data using our method effectively facilitates real-world learning tasks.
ScribbleBox: Interactive Annotation Framework for Video Object Segmentation
Our approach tolerates a modest amount of noise in the box placements, thus typically only a few clicks are needed to annotate tracked boxes to a sufficient accuracy.
Beyond Fixed Grid: Learning Geometric Image Representation with a Deformable Grid
We also utilize DefGrid at the output layers for the task of object mask annotation, and show that reasoning about object boundaries on our predicted polygonal grid leads to more accurate results over existing pixel-wise and curve-based approaches.
Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation
In Meta-Sim2, we aim to learn the scene structure in addition to parameters, which is a challenging problem due to its discrete nature.
Learning to Generate Diverse Dance Motions with Transformer
We also introduce new evaluation metrics for the quality of synthesized dance motions, and demonstrate that our system can outperform state-of-the-art methods.
Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D
By training on the entire camera rig, we provide evidence that our model is able to learn not only how to represent images but how to fuse predictions from all cameras into a single cohesive representation of the scene while being robust to calibration error.
Learning to Simulate Dynamic Environments with GameGAN
Simulation is a crucial component of any robotic system.
Efficient and Information-Preserving Future Frame Prediction and Beyond
Applying resolution-preserving blocks is a common practice to maximize information preservation in video prediction, yet their high memory consumption greatly limits their application scenarios.
Ranked #13 on
Video Prediction
on Moving MNIST
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines
Our dataset features 55 hours of video consisting of 11. 5M frames, which we densely labelled for a total of 39. 6K action segments and 454. 2K object bounding boxes.
Learning to Evaluate Perception Models Using Planner-Centric Metrics
The downside of these metrics is that, at worst, they penalize all incorrect detections equally without conditioning on the task or scene, and at best, heuristics need to be chosen to ensure that different mistakes count differently.
Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
NDS consists of a dataserver which indexes several large popular image datasets, and aims to recommend data to a client, an end-user with a target application with its own small labeled dataset.
The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries
Understanding stories is a challenging reading comprehension problem for machines as it requires reading a large volume of text and following long-range dependencies.
Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research
We present Kaolin, a PyTorch library aiming to accelerate 3D deep learning research.
CrevNet: Conditionally Reversible Video Prediction
Applying resolution-preserving blocks is a common practice to maximize information preservation in video prediction, yet their high memory consumption greatly limits their application scenarios.
Neural Turtle Graphics for Modeling City Road Layouts
We propose Neural Turtle Graphics (NTG), a novel generative model for spatial graphs, and demonstrate its applications in modeling city road layouts.
DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation
In practice, it performs similarly to the Hungarian algorithm during inference.
One-shot visual object segmentation
Semantic Segmentation
+1
A Theoretical Analysis of the Number of Shots in Few-Shot Learning
We introduce a theoretical analysis of the impact of the shot number on Prototypical Networks, a state-of-the-art few-shot classification method.
Video Face Clustering with Unknown Number of Clusters
Understanding videos such as TV series and movies requires analyzing who the characters are and what they are doing.
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
Many machine learning models operate on images, but ignore the fact that images are 2D projections formed by 3D geometry interacting with light, in a process called rendering.
Ranked #4 on
Single-View 3D Reconstruction
on ShapeNet
Gated-SCNN: Gated Shape CNNs for Semantic Segmentation
Here, we propose a new two-stream CNN architecture for semantic segmentation that explicitly wires shape information as a separate processing branch, i. e. shape stream, that processes information in parallel to the classical stream.
Ranked #18 on
Semantic Segmentation
on Cityscapes test
Neural Graph Evolution: Towards Efficient Automatic Robot Design
To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space.
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis
Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices.
DARNet: Deep Active Ray Network for Building Segmentation
In this paper, we propose a Deep Active Ray Network (DARNet) for automatic building segmentation.
Neural Graph Evolution: Automatic Robot Design
To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space.
ACTRCE: Augmenting Experience via Teacher’s Advice
Sparse reward is one of the most challenging problems in reinforcement learning (RL).
Meta-Sim: Learning to Generate Synthetic Datasets
Training models to high-end performance requires availability of large labeled datasets, which are expensive to get.
Devil is in the Edges: Learning Semantic Boundaries from Noisy Annotations
We further reason about true object boundaries during training using a level set formulation, which allows the network to learn from misaligned labels in an end-to-end fashion.
Action Recognition from Single Timestamp Supervision in Untrimmed Videos
We propose a method that is supervised by single timestamps located around each action instance, in untrimmed videos.
Mimicking the In-Camera Color Pipeline for Camera-Aware Object Compositing
We present a method for compositing virtual objects into a photograph such that the object colors appear to have been processed by the photo's camera imaging pipeline.
Fast Interactive Object Annotation with Curve-GCN
Our model runs at 29. 3ms in automatic, and 2. 6ms in interactive mode, making it 10x and 100x faster than Polygon-RNN++.
ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning
We first analyze the differences among goal representation, and show that ACTRCE can efficiently solve difficult reinforcement learning problems in challenging 3D navigation tasks, whereas HER with non-language goal representation failed to learn.
SurfConv: Bridging 3D and 2D Convolution for RGBD Images
On the other hand, 3D convolution wastes a large amount of memory on mostly unoccupied 3D space, which consists of only the surface visible to the sensor.
A Face-to-Face Neural Conversation Model
The decoder consists of two layers, where the lower layer aims at generating the verbal response and coarse facial expressions, while the second layer fills in the subtle gestures, making the generated output more smooth and natural.
Learning to Caption Images through a Lifetime by Asking Questions
In order to bring artificial agents into our lives, we will need to go beyond supervised learning on closed datasets to having the ability to continuously expand knowledge.
A Neural Compositional Paradigm for Image Captioning
Mainstream captioning models often follow a sequential structure to generate captions, leading to issues such as introduction of irrelevant semantics, lack of diversity in the generated captions, and inadequate generalization performance.
Pose Estimation for Objects with Rotational Symmetry
In particular, our aim is to infer poses for objects not seen at training time, but for which their 3D CAD models are available at test time.
VirtualHome: Simulating Household Activities via Programs
We then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment.
Color Sails: Discrete-Continuous Palettes for Deep Color Exploration
Then, the user can adjust color sail parameters to change the base colors, their blending behavior and the number of colors, exploring a wide range of options for the original design.
Visual Reasoning by Progressive Module Networks
Thus, a module for a new task learns to query existing modules and composes their outputs in order to produce its own output.
Now You Shake Me: Towards Automatic 4D Cinema
We are interested in enabling automatic 4D cinema by parsing physical and special effects from untrimmed movies.
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset
First-person vision is gaining interest as it offers a unique viewpoint on people's interaction with objects, their attention, and even intention.
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
Manually labeling datasets with object masks is extremely time consuming.
NerveNet: Learning Structured Policy with Graph Neural Networks
We address the problem of learning structured policies for continuous control.
Learning to Act Properly: Predicting and Explaining Affordances from Images
We address the problem of affordance reasoning in diverse scenes that appear in the real world.
MovieGraphs: Towards Understanding Human-Centric Situations from Videos
Towards this goal, we introduce a novel dataset called MovieGraphs which provides detailed, graph-based annotations of social situations depicted in movie clips.
Teaching Machines to Describe Images with Natural Language Feedback
Robots will eventually be part of every household.
Be Your Own Prada: Fashion Synthesis with Structural Coherence
In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.
SGN: Sequential Grouping Networks for Instance Segmentation
By exploiting two-directional information, the second network groups horizontal and vertical lines into connected components.
3D Graph Neural Networks for RGBD Semantic Segmentation
Each node in the graph corresponds to a set of points and is associated with a hidden representation vector initialized with an appearance feature extracted by a unary CNN from 2D images.
Ranked #21 on
Semantic Segmentation
on SUN-RGBD
Situation Recognition with Graph Neural Networks
We address the problem of recognizing situations in images.
Ranked #8 on
Grounded Situation Recognition
on SWiG
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives
We present a new technique for learning visual-semantic embeddings for cross-modal retrieval.
Ranked #19 on
Cross-Modal Retrieval
on Flickr30k
Scene Parsing Through ADE20K Dataset
A novel network design called Cascade Segmentation Module is proposed to parse a scene into stuff, objects, and object parts in a cascade and improve over the baselines.
Sports Field Localization via Deep Structured Models
In this work, we propose a novel way of efficiently localizing a sports field from a single broadcast image of the game.
Teaching Machines to Describe Images via Natural Language Feedback
Robots will eventually be part of every household.
Annotating Object Instances with a Polygon-RNN
We show that our approach speeds up the annotation process by a factor of 4. 7 across all classes in Cityscapes, while achieving 78. 4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators.
Open Vocabulary Scene Parsing
Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets.
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Despite the substantial progress in recent years, the image captioning techniques are still far from being perfect. Sentences produced by existing methods, e. g. those based on RNNs, are often overly rigid and lacking in variability.
TorontoCity: Seeing the World with a Million Eyes
In this paper we introduce the TorontoCity benchmark, which covers the full greater Toronto area (GTA) with 712. 5 $km^2$ of land, 8439 $km$ of road and around 400, 000 buildings.
Proximal Deep Structured Models
Many problems in real-world applications involve predicting continuous-valued random variables that are statistically related.
Song From PI: A Musically Plausible Network for Pop Music Generation
We present a novel framework for generating pop music.
Efficient Summarization with Read-Again and Copy Mechanism
Towards this goal, we first introduce a simple mechanism that first reads the input sequence before committing to a representation of each word.
3D Object Proposals using Stereo Imagery for Accurate Object Class Detection
We then exploit a CNN on top of these proposals to perform object detection.
Semantic Understanding of Scenes through the ADE20K Dataset
Scene parsing, or recognizing and segmenting objects and stuff in an image, is one of the key problems in computer vision.
Find your Way by Observing the Sun and Other Semantic Cues
In this paper we present a robust, efficient and affordable approach to self-localization which does not require neither GPS nor knowledge about the appearance of the world.
HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images
In this paper we present an approach to enhance existing maps with fine grained segmentation categories such as parking spots and sidewalk, as well as the number and location of road lanes.
Monocular 3D Object Detection for Autonomous Driving
The focus of this paper is on proposal generation.
Ranked #8 on
Vehicle Pose Estimation
on KITTI Cars Hard
Soccer Field Localization from a Single Image
In this work, we propose a novel way of efficiently localizing a soccer field from a single broadcast image of the game.
Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs
Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving.
MovieQA: Understanding Stories in Movies through Question-Answering
We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text.
3D Object Proposals for Accurate Object Class Detection
The goal of this paper is to generate high-quality 3D object proposals in the context of autonomous driving.
Ranked #10 on
Vehicle Pose Estimation
on KITTI Cars Hard
Lost Shopping! Monocular Localization in Large Indoor Spaces
In this paper we propose a novel approach to localization in very large indoor spaces (i. e., 200+ store shopping malls) that takes a single image and a floor plan of the environment as input.
Learning to Combine Mid-Level Cues for Object Proposal Generation
In this paper, we introduce Parametric Min-Loss (PML), a novel structured learning framework for parametric energy functions.
Enhancing Road Maps by Parsing Aerial Images Around the World
In recent years, contextual models that exploit maps have been shown to be very effective for many recognition and localization tasks.
Order-Embeddings of Images and Language
Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images.
Ranked #89 on
Natural Language Inference
on SNLI
Skip-Thought Vectors
The end result is an off-the-shelf encoder that can produce highly generic sentence representations that are robust and perform well in practice.
Ranked #2 on
Semantic Similarity
on SICK
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
Books are a rich source of both fine-grained information, how a character, an object or a scene looks like, as well as high-level semantics, what someone is thinking, feeling and how these states evolve through a story.
Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions
One of the main challenges in Zero-Shot Learning of visual categories is gathering semantic attributes to accompany images.
Holistic 3D Scene Understanding From a Single Geo-Tagged Image
In this paper we are interested in exploiting geographic priors to help outdoor scene understanding.
Rent3D: Floor-Plan Priors for Monocular Layout Estimation
What sets us apart from past work in layout estimation is the use of floor plans as a source of prior knowledge, as well as localization of each image within a bigger space (apartment).
Real-Time Coarse-to-Fine Topologically Preserving Segmentation
In this paper, we tackle the problem of unsupervised segmentation in the form of superpixels.
Monocular Object Instance Segmentation and Depth Ordering with CNNs
In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image.
Generating Multi-Sentence Lingual Descriptions of Indoor Scenes
This paper proposes a novel framework for generating lingual descriptions of indoor scenes.
segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
In this paper, we propose an approach that exploits object segmentation in order to improve the accuracy of object detection.
A Framework for Symmetric Part Detection in Cluttered Scenes
The role of symmetry in computer vision has waxed and waned in importance during the evolution of the field from its earliest days.
Neuroaesthetics in Fashion: Modeling the Perception of Fashionability
Importantly, our model is able to give rich feedback back to the user, conveying which garments or even scenery she/he should change in order to improve fashionability.
Learning a Hierarchical Compositional Shape Vocabulary for Multi-class Object Representation
At the top-level of the vocabulary, the compositions are sufficiently large and complex to represent the whole shapes of the objects.
Video In Sentences Out
We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it.
Human-Machine CRFs for Identifying Bottlenecks in Holistic Scene Understanding
Recent trends in image understanding have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning, and local appearance based classifiers.
Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
Our model automatically decouples the holistic object or body parts from the model when they are hard to detect.
What are You Talking About? Text-to-Image Coreference
In this paper we exploit natural sentential descriptions of RGB-D scenes in order to improve 3D semantic parsing.
Beat the MTurkers: Automatic Image Labeling from Weak 3D Supervision
Labeling large-scale datasets with very accurate object segmentations is an elaborate task that requires a high degree of quality control and a budget of tens or hundreds of thousands of dollars.
Visual Semantic Search: Retrieving Videos via Complex Textual Queries
In this paper, we tackle the problem of retrieving videos using complex natural language queries.
The Role of Context for Object Detection and Semantic Segmentation in the Wild
In this paper we study the role of context in existing state-of-the-art detection and segmentation approaches.
Bottom-Up Segmentation for Top-Down Detection
When employing the parts, we outperform the original DPM [14] in 19 out of 20 classes, achieving an improvement of 8% AP.
Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs
Recent trends in semantic image segmentation have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning.
A Sentence Is Worth a Thousand Pixels
We are interested in holistic scene understanding where images are accompanied with text in the form of complex sentential descriptions.
3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model
We demonstrate the effectiveness of our approach in indoor and outdoor scenarios, and show that our approach outperforms the state-of-the-art in both 2D[Felz09] and 3D object detection[Hedau12].
Evaluating multi-class learning strategies in a generative hierarchical framework for object detection
We explore and compare their computational behavior (space and time) and detection performance as a function of the number of learned classes on several recognition data sets.
