Search Results for author:
Found 181 papers, 42 papers with code
GoRela: Go Relative for Viewpoint-Invariant Motion Forecasting
However, this approach is computationally expensive for multi-agent prediction as inference needs to be run for each agent.
Virtual Correspondence: Humans as a Cue for Extreme-View Geometry
Similar to classic correspondences, VCs conform with epipolar geometry; unlike classic correspondences, VCs do not need to be co-visible across views.
NP-DRAW: A Non-Parametric Structured Latent Variable Model for Image Generation
1) We propose a non-parametric prior distribution over the appearance of image parts so that the latent variable ``what-to-draw'' per step becomes a categorical random variable.
Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
As data collection is often significantly cheaper than labeling in this domain, the decision of which subset of examples to label can have a profound impact on model performance.
IntentNet: Learning to Predict Intention from Raw Sensor Data
In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants.
Deep Feedback Inverse Problem Solver
Specifically, at each iteration, the neural network takes the feedback as input and outputs an update on the current estimation.
Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wild
Reconstructing high-quality 3D objects from sparse, partial observations from a single view is of crucial importance for various applications in computer vision, robotics, and graphics.
Deep Structured Reactive Planning
An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene.
Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving
In this paper, we introduce a non-parametric memory representation for spatio-temporal segmentation that captures the local space and time around an autonomous vehicle (AV).
MP3: A Unified Model to Map, Perceive, Predict and Plan
High-definition maps (HD maps) are a key component of most modern self-driving systems due to their valuable semantic and geometric information.
PLUMENet: Efficient 3D Object Detection from Stereo Images
In this paper we propose a model that unifies these two tasks and performs them in the same metric space.
Network Automatic Pruning: Start NAP and Take a Nap
This process is typically time-consuming and requires expert knowledge to achieve good results.
End-to-end Interpretable Neural Motion Planner
In this paper, we propose a neural motion planner (NMP) for learning to drive autonomously in complex urban scenarios that include traffic-light handling, yielding, and interactions with multiple road-users.
LaneRCNN: Distributed Representations for Graph-Centric Motion Forecasting
In this paper, we propose LaneRCNN, a graph-centric motion forecasting model.
Ranked #149 on
Motion Forecasting
on Argoverse CVPR 2020
Deep Multi-Task Learning for Joint Localization, Perception, and Prediction
Over the last few years, we have witnessed tremendous progress on many subtasks of autonomous driving, including perception, motion forecasting, and motion planning.
Deep Parametric Continuous Convolutional Neural Networks
Standard convolutional neural networks assume a grid structured input is available and exploit discrete convolutions as their fundamental building blocks.
Ranked #18 on
Semantic Segmentation
on S3DIS Area5
(mAcc metric)
Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving
Yet, there have been limited studies on the adversarial robustness of multi-modal models that fuse LiDAR features with image features.
TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors
We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
Adversarial Attacks On Multi-Agent Communication
Growing at a fast pace, modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems.
Asynchronous Multi-View SLAM
Existing multi-camera SLAM systems assume synchronized shutters for all cameras, which is often not the case in practice.
S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
Constructing and animating humans is an important component for building virtual worlds in a wide variety of applications such as virtual reality or robotics testing in simulation.
Auto4D: Learning to Label 4D Objects from Sequential Point Clouds
The key idea is to decompose the 4D object label into two parts: the object size in 3D that's fixed through time for rigid objects, and the motion path describing the evolution of the object's pose through time.
Cost-Efficient Online Hyperparameter Optimization
Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters.
LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
In this paper, we present LookOut, a novel autonomy system that perceives the environment, predicts a diverse set of futures of how the scene might unroll and estimates the trajectory of the SDV by optimizing a set of contingency plans over these future realizations.
SceneGen: Learning to Generate Realistic Traffic Scenes
Existing methods typically insert actors into the scene according to a set of hand-crafted heuristics and are limited in their ability to model the true complexity and diversity of real traffic scenes, thus inducing a content gap between synthesized traffic scenes versus real ones.
GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
Scalable sensor simulation is an important yet challenging open problem for safety-critical domains such as self-driving.
VideoClick: Video Object Segmentation with a Single Click
Towards this goal, in this paper we propose a bottom up approach where given a single click for each object in a video, we obtain the segmentation masks of these objects in the full video.
AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
Importantly, by simulating directly from sensor data, we obtain adversarial scenarios that are safety-critical for the full autonomy stack.
Self-Supervised Representation Learning from Flow Equivariance
Motivated by this ability, we present a new self-supervised learning representation framework that can be directly deployed on a video stream of complex scenes with many moving objects.
Diverse Complexity Measures for Dataset Curation in Self-driving
Our experiments on a wide range of tasks and models show that the proposed curation pipeline is able to select datasets that lead to better generalization and higher performance.
Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
On two large-scale real-world datasets, nuScenes and ATG4D, we showcase that our scene-occupancy predictions are more accurate and better calibrated than those from state-of-the-art motion forecasting methods, while also matching their performance in pedestrian motion forecasting metrics.
Pit30M: A Benchmark for Global Localization in the Age of Self-Driving Cars
We are interested in understanding whether retrieval-based localization approaches are good enough in the context of self-driving vehicles.
Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
In this paper we propose a novel deep neural network that is able to jointly reason about 3D detection, tracking and motion forecasting given data captured by a 3D sensor.
Multi-Task Multi-Sensor Fusion for 3D Object Detection
In this paper we propose to exploit multiple related tasks for accurate multi-sensor 3D object detection.
Ranked #13 on
3D Object Detection
on KITTI Cars Easy
Learning Joint 2D-3D Representations for Depth Completion
In this paper, we tackle the problem of depth completion from RGBD data.
DAGMapper: Learning to Map by Discovering Lane Topology
One of the fundamental challenges to scale self-driving is being able to create accurate high definition maps (HD maps) with low cost.
Hierarchical Recurrent Attention Networks for Structured Online Maps
In this paper, we tackle the problem of online road network extraction from sparse 3D point clouds.
End-to-End Deep Structured Models for Drawing Crosswalks
In this paper we address the problem of detecting crosswalks from LiDAR and camera imagery.
Convolutional Recurrent Network for Road Boundary Extraction
Creating high definition maps that contain precise information of static elements of the scene is of utmost importance for enabling self driving cars to drive safely.
HDNET: Exploiting HD Maps for 3D Object Detection
In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors.
Learning to Localize Using a LiDAR Intensity Map
In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars.
Learning to Localize Through Compressed Binary Maps
One of the main difficulties of scaling current localization systems to large environments is the on-board storage required for the maps.
Deep Continuous Fusion for Multi-Sensor 3D Object Detection
In this paper, we propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization.
A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks
In this paper, we derive generalization bounds for the two primary classes of graph neural networks (GNNs), namely graph convolutional networks (GCNs) and message passing GNNs (MPGNNs), via a PAC-Bayesian approach.
GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation
Note that GeoNet++ is generic and can be used in other depth/normal prediction frameworks to improve the quality of 3D reconstruction and pixel-wise accuracy of depth and surface normals.
Recovering and Simulating Pedestrians in the Wild
We then incorporate the reconstructed pedestrian assets bank in a realistic LiDAR simulation system by performing motion retargeting, and show that the simulated LiDAR data can be used to significantly reduce the amount of annotated real-world data required for visual perception tasks.
MuSCLE: Multi Sweep Compression of LiDAR using Deep Entropy Models
Our model exploits spatio-temporal relationships across multiple LiDAR sweeps to reduce the bitrate of both geometry and intensity values.
Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
In this paper, we tackle the problem of spatio-temporal tagging of self-driving scenes from raw sensor data.
StrObe: Streaming Object Detection from LiDAR Packets
In this paper we propose StrObe, a novel approach that minimizes latency by ingesting LiDAR packets and emitting a stream of detections without waiting for the full sweep to be built.
Learning to Communicate and Correct Pose Errors
Learned communication makes multi-agent systems more effective by aggregating distributed information.
Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Driving
In this paper, we propose an end-to-end self-driving network featuring a sparse attention module that learns to automatically attend to important regions of the input.
Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks
Compressing large neural networks is an important step for their deployment in resource-constrained computational platforms.
LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion
In this paper, we present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps.
Weakly-supervised 3D Shape Completion in the Wild
3D shape completion for real data is important but challenging, since partial point clouds acquired by real-world sensors are usually sparse, noisy and unaligned.
V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
In this paper, we explore the use of vehicle-to-vehicle (V2V) communication to improve the perception and motion forecasting performance of self-driving vehicles.
Ranked #1 on
3D Object Detection
on OPV2V
Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations
In this paper we propose a novel end-to-end learnable network that performs joint perception, prediction and motion planning for self-driving vehicles and produces interpretable intermediate representations.
End-to-end Contextual Perception and Prediction with Interaction Transformer
We show that our approach can outperform the state-of-the-art on both datasets.
DSDNet: Deep Structured self-Driving Network
In this paper, we propose the Deep Structured self-Driving Network (DSDNet), which performs object detection, motion prediction, and motion planning with a single neural network.
Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction
We present a novel method for testing the safety of self-driving vehicles in simulation.
LoCo: Local Contrastive Representation Learning
Deep neural nets typically perform end-to-end backpropagation to learn the weights, a procedure that creates synchronization constraints in the weight update step across layers and is not biologically plausible.
LevelSet R-CNN: A Deep Variational Method for Instance Segmentation
Obtaining precise instance segmentation masks is of high importance in many modern applications such as robotic manipulation and autonomous driving.
RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects
We tackle the problem of exploiting Radar for perception in the context of self-driving as Radar provides complementary information to other sensors such as LiDAR or cameras in the form of Doppler velocity.
Learning Lane Graph Representations for Motion Forecasting
We propose a motion forecasting model that exploits a novel structured map representation as well as actor-map interactions.
Hierarchical Verification for Adversarial Robustness
We show that, under certain conditions on the algorithm parameters, LayerCert provably reduces the number and size of the convex programs that one needs to solve compared to GeoCert.
Implicit Latent Variable Model for Scene-Consistent Motion Forecasting
In order to plan a safe maneuver an autonomous vehicle must accurately perceive its environment, and understand the interactions among traffic participants.
Multi-Agent Routing Value Iteration Network
In this paper we tackle the problem of routing multiple agents in a coordinated manner.
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World
We first utilize ray casting over the 3D scene and then use a deep neural network to produce deviations from the physics-based simulation, producing realistic LiDAR point clouds.
The Importance of Prior Knowledge in Precise Multimodal Prediction
Towards this goal, we design a framework that leverages REINFORCE to incorporate non-differentiable priors over sample trajectories from a probabilistic model, thus optimizing the whole distribution.
PnPNet: End-to-End Perception and Prediction with Tracking in the Loop
We tackle the problem of joint perception and motion forecasting in the context of self-driving vehicles.
ShapeAdv: Generating Shape-Aware Adversarial 3D Point Clouds
Our shape-aware adversarial attacks are orthogonal to existing point cloud based attacks and shed light on the vulnerability of 3D deep neural networks.
OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression
We present a novel deep compression algorithm to reduce the memory footprint of LiDAR point clouds.
Physically Realizable Adversarial Examples for LiDAR Object Detection
Modern autonomous driving systems rely heavily on deep learning models to process point cloud sensory data; meanwhile, deep models have been shown to be susceptible to adversarial attacks with visually imperceptible perturbations.
Dense RepPoints: Representing Visual Objects with Dense Point Sets
We present a new object representation, called Dense RepPoints, that utilizes a large set of points to describe an object at multiple levels, including both box level and pixel level.
PolyTransform: Deep Polygon Transformer for Instance Segmentation
In this paper, we propose PolyTransform, a novel instance segmentation algorithm that produces precise, geometry-preserving masks by combining the strengths of prevailing segmentation approaches and modern polygon-based methods.
Ranked #1 on
Instance Segmentation
on Cityscapes test
(using extra training data)
Identifying Unknown Instances for Autonomous Driving
In the past few years, we have seen great progress in perception algorithms, particular through the use of deep learning.
Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
A graph neural network then iteratively updates the actor states via a message passing process.
Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction
Particularly difficult is the prediction of human behavior.
Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles
The motion planners used in self-driving vehicles need to generate trajectories that are safe, comfortable, and obey the traffic rules.
Learning to Remember from a Multi-Task Teacher
Recent studies on catastrophic forgetting during sequential learning typically focus on fixing the accuracy of the predictions for a previously learned task.
Efficient Graph Generation with Graph Recurrent Attention Networks
Our model generates graphs one block of nodes and associated edges at a time.
DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation
In practice, it performs similarly to the Hungarian algorithm during inference.
One-shot visual object segmentation
Semantic Segmentation
+1
DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch
Our goal is to significantly speed up the runtime of current state-of-the-art stereo algorithms to enable real-time inference.
DSIC: Deep Stereo Image Compression
In this paper we tackle the problem of stereo image compression, and leverage the fact that the two images have overlapping fields of view to further compress the representations.
Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization
In this paper we propose a novel semantic localization algorithm that exploits multiple sensors and has precision on the order of a few centimeters.
Deformable Filter Convolution for Point Cloud Reasoning
Point clouds are the native output of many real-world 3D sensors.
DARNet: Deep Active Ray Network for Building Segmentation
In this paper, we propose a Deep Active Ray Network (DARNet) for automatic building segmentation.
Deep Multi-Sensor Lane Detection
Reliable and accurate lane detection has been a long-standing problem in the field of autonomous driving.
DeepSignals: Predicting Intent of Drivers Through Visual Signals
Detecting the intention of drivers is an essential task in self-driving, necessary to anticipate sudden events like lane changes and stops.
Dimensionality Reduction for Representing the Knowledge of Probabilistic Models
Most deep learning models rely on expressive high-dimensional representations to achieve good performance on tasks such as classification.
Deep Rigid Instance Scene Flow
In this paper we tackle the problem of scene flow estimation in the context of self-driving.
PIXOR: Real-time 3D Object Detection from Point Clouds
Existing approaches are, however, expensive in computation due to high dimensionality of point clouds.
UPSNet: A Unified Panoptic Segmentation Network
More importantly, we introduce a parameter-free panoptic head which solves the panoptic segmentation via pixel-wise classification.
Ranked #3 on
Panoptic Segmentation
on KITTI Panoptic Segmentation
LanczosNet: Multi-Scale Deep Graph Convolutional Networks
We propose the Lanczos network (LanczosNet), which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution.
SurfConv: Bridging 3D and 2D Convolution for RGBD Images
On the other hand, 3D convolution wastes a large amount of memory on mostly unoccupied 3D space, which consists of only the surface visible to the sensor.
Graph HyperNetworks for Neural Architecture Search
Neural architecture search (NAS) automatically finds the best task-specific neural network topology, outperforming many manual architecture designs.
MLPrune: Multi-Layer Pruning for Automated Neural Network Compression
Model compression can significantly reduce the computation and memory footprint of large neural networks.
Neural Guided Constraint Logic Programming for Program Synthesis
Synthesizing programs using example input/outputs is a classic problem in artificial intelligence.
Single Image Intrinsic Decomposition without a Single Intrinsic Image
At inference time, our model can be easily reduced to a single stream module that performs intrinsic decomposition on a single input image.
End-to-end Learning of Multi-sensor 3D Tracking by Detection
In this paper we propose a novel approach to tracking by detection that can exploit both cameras as well as LIDAR data to produce very accurate 3D trajectories.
Ranked #5 on
3D Multi-Object Tracking
on KITTI
GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation
In this paper, we propose Geometric Neural Network (GeoNet) to jointly predict depth and surface normal maps from a single image.
Matching Adversarial Networks
We argue that the main difficulty of applying CGANs to supervised tasks is that the generator training consists of optimizing a loss function that does not depend directly on the ground truth labels.
Learning to Reweight Examples for Robust Deep Learning
Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns.
Inference in Probabilistic Graphical Models by Graph Neural Networks
Message-passing algorithms, such as belief propagation, are a natural way to disseminate evidence amongst correlated variables while exploiting the graph structure, but these algorithms can struggle when the conditional dependency graphs contain loops.
Graph Partition Neural Networks for Semi-Supervised Classification
We present graph partition neural networks (GPNN), an extension of graph neural networks (GNNs) able to handle extremely large graphs.
Learning deep structured active contours end-to-end
The world is covered with millions of buildings, and precisely knowing each instance's position and extents is vital to a multitude of applications.
Reviving and Improving Recurrent Back-Propagation
We examine all RBP variants along with BPTT and TBPTT in three different application domains: associative memory with continuous Hopfield networks, document classification in citation networks using graph neural networks and hyperparameter optimization for fully connected networks.
SBNet: Sparse Blocks Network for Fast Inference
Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications.
Be Your Own Prada: Fashion Synthesis with Structural Coherence
In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.
DeepRoadMapper: Extracting Road Topology From Aerial Images
In contrast, in this paper we propose an approach that directly estimates road topology from aerial images.
3D Graph Neural Networks for RGBD Semantic Segmentation
Each node in the graph corresponds to a set of points and is associated with a hidden representation vector initialized with an appearance feature extracted by a unary CNN from 2D images.
Ranked #21 on
Semantic Segmentation
on SUN-RGBD
SGN: Sequential Grouping Networks for Instance Segmentation
By exploiting two-directional information, the second network groups horizontal and vertical lines into connected components.
Situation Recognition with Graph Neural Networks
We address the problem of recognizing situations in images.
Ranked #8 on
Grounded Situation Recognition
on SWiG
Deep Spectral Clustering Learning
We derive a closed-form expression for the gradient that is efficient to compute: the complexity to compute the gradient is linear in the size of the training mini-batch and quadratic in the representation dimensionality.
The Reversible Residual Network: Backpropagation Without Storing Activations
Deep residual networks (ResNets) have significantly pushed forward the state-of-the-art on image classification, increasing in performance as networks grow both deeper and wider.
Few-Shot Learning Through an Information Retrieval Lens
Few-shot learning refers to understanding new concepts from only a few examples.
Sports Field Localization via Deep Structured Models
In this work, we propose a novel way of efficiently localizing a sports field from a single broadcast image of the game.
Efficient Multiple Instance Metric Learning Using Weakly Supervised Data
Classic approaches alternate the optimization over the learned metric and the assignment of similar instances.
Annotating Object Instances with a Polygon-RNN
We show that our approach speeds up the annotation process by a factor of 4. 7 across all classes in Cityscapes, while achieving 78. 4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators.
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Despite the substantial progress in recent years, the image captioning techniques are still far from being perfect. Sentences produced by existing methods, e. g. those based on RNNs, are often overly rigid and lacking in variability.
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
We study characteristics of receptive fields of units in deep convolutional networks.
MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
While most approaches to semantic reasoning have focused on improving performance, in this paper we argue that computational times are very important in order to enable real time applications such as autonomous driving.
Proximal Deep Structured Models
Many problems in real-world applications involve predicting continuous-valued random variables that are statistically related.
TorontoCity: Seeing the World with a Million Eyes
In this paper we introduce the TorontoCity benchmark, which covers the full greater Toronto area (GTA) with 712. 5 $km^2$ of land, 8439 $km$ of road and around 400, 000 buildings.
Learning Deep Parsimonious Representations
In this paper we aim at facilitating generalization for deep networks while supporting interpretability of the learned representations.
Deep Watershed Transform for Instance Segmentation
Most contemporary approaches to instance segmentation use complex pipelines involving conditional random fields, recurrent neural networks, object proposals, or template matching schemes.
Ranked #10 on
Instance Segmentation
on Cityscapes test
Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes
On the other hand, layer normalization normalizes the activations across all activities within a layer.
Song From PI: A Musically Plausible Network for Pop Music Generation
We present a novel framework for generating pop music.
Efficient Summarization with Read-Again and Copy Mechanism
Towards this goal, we first introduce a simple mechanism that first reads the input sequence before committing to a representation of each word.
3D Object Proposals using Stereo Imagery for Accurate Object Class Detection
We then exploit a CNN on top of these proposals to perform object detection.
Find your Way by Observing the Sun and Other Semantic Cues
In this paper we present a robust, efficient and affordable approach to self-localization which does not require neither GPS nor knowledge about the appearance of the world.
Monocular 3D Object Detection for Autonomous Driving
The focus of this paper is on proposal generation.
Ranked #8 on
Vehicle Pose Estimation
on KITTI Cars Hard
HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images
In this paper we present an approach to enhance existing maps with fine grained segmentation categories such as parking spots and sidewalk, as well as the number and location of road lanes.
Efficient Deep Learning for Stereo Matching
In the past year, convolutional neural networks have been shown to perform extremely well for stereo estimation.
Soccer Field Localization from a Single Image
In this work, we propose a novel way of efficiently localizing a soccer field from a single broadcast image of the game.
Exploiting Semantic Information and Deep Matching for Optical Flow
We tackle the problem of estimating optical flow from a monocular camera in the context of autonomous driving.
Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs
Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving.
MovieQA: Understanding Stories in Movies through Question-Answering
We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text.
Enhancing Road Maps by Parsing Aerial Images Around the World
In recent years, contextual models that exploit maps have been shown to be very effective for many recognition and localization tasks.
Lost Shopping! Monocular Localization in Large Indoor Spaces
In this paper we propose a novel approach to localization in very large indoor spaces (i. e., 200+ store shopping malls) that takes a single image and a floor plan of the environment as input.
3D Object Proposals for Accurate Object Class Detection
The goal of this paper is to generate high-quality 3D object proposals in the context of autonomous driving.
Ranked #10 on
Vehicle Pose Estimation
on KITTI Cars Hard
Order-Embeddings of Images and Language
Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images.
Ranked #89 on
Natural Language Inference
on SNLI
Training Deep Neural Networks via Direct Loss Minimization
Supervised training of deep neural nets typically relies on minimizing cross-entropy.
Reduced-Precision Strategies for Bounded Memory in Deep Neural Nets
A diverse set of CNNs is analyzed showing that compared to a conventional implementation using a 32-bit floating-point representation for all layers, and with less than 1% loss in relative accuracy, the data footprint required by these networks can be reduced by an average of 74% and up to 92%.
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
Books are a rich source of both fine-grained information, how a character, an object or a scene looks like, as well as high-level semantics, what someone is thinking, feeling and how these states evolve through a story.
Skip-Thought Vectors
The end result is an off-the-shelf encoder that can produce highly generic sentence representations that are robust and perform well in practice.
Ranked #2 on
Semantic Similarity
on SICK
Learning to Segment Under Various Forms of Weak Supervision
Despite the promising performance of conventional fully supervised algorithms, semantic segmentation has remained an important, yet challenging task.
Rent3D: Floor-Plan Priors for Monocular Layout Estimation
What sets us apart from past work in layout estimation is the use of floor plans as a source of prior knowledge, as well as localization of each image within a bigger space (apartment).
Holistic 3D Scene Understanding From a Single Geo-Tagged Image
In this paper we are interested in exploiting geographic priors to help outdoor scene understanding.
Real-Time Coarse-to-Fine Topologically Preserving Segmentation
In this paper, we tackle the problem of unsupervised segmentation in the form of superpixels.
Monocular Object Instance Segmentation and Depth Ordering with CNNs
In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image.
Fully Connected Deep Structured Networks
Convolutional neural networks with many layers have recently been shown to achieve excellent results on many high-level tasks such as image classification, object detection and more recently also semantic segmentation.
Generating Multi-Sentence Lingual Descriptions of Indoor Scenes
This paper proposes a novel framework for generating lingual descriptions of indoor scenes.
segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
In this paper, we propose an approach that exploits object segmentation in order to improve the accuracy of object detection.
Neuroaesthetics in Fashion: Modeling the Perception of Fashionability
Importantly, our model is able to give rich feedback back to the user, conveying which garments or even scenery she/he should change in order to improve fashionability.
Message Passing Inference for Large Scale Graphical Models with High Order Potentials
To keep up with the Big Data challenge, parallelized algorithms based on dual decomposition have been proposed to perform inference in Markov random fields.
Efficient Inference of Continuous Markov Random Fields with Polynomial Potentials
In this paper, we prove that every multivariate polynomial with even degree can be decomposed into a sum of convex and concave polynomials.
FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation
One of the most popular approaches to multi-target tracking is tracking-by-detection.
Ranked #23 on
Multiple Object Tracking
on KITTI Tracking test
Learning Deep Structured Models
Towards this goal, we propose a training algorithm that is able to learn structured models jointly with deep features that form the MRF potentials.
Human-Machine CRFs for Identifying Bottlenecks in Holistic Scene Understanding
Recent trends in image understanding have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning, and local appearance based classifiers.
Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
Our model automatically decouples the holistic object or body parts from the model when they are hard to detect.
Tell Me What You See and I will Show You Where It Is
We tackle the problem of weakly labeled semantic segmentation, where the only source of annotation are image tags encoding which classes are present in the scene.
What are You Talking About? Text-to-Image Coreference
In this paper we exploit natural sentential descriptions of RGB-D scenes in order to improve 3D semantic parsing.
The Role of Context for Object Detection and Semantic Segmentation in the Wild
In this paper we study the role of context in existing state-of-the-art detection and segmentation approaches.
Visual Semantic Search: Retrieving Videos via Complex Textual Queries
In this paper, we tackle the problem of retrieving videos using complex natural language queries.
Beat the MTurkers: Automatic Image Labeling from Weak 3D Supervision
Labeling large-scale datasets with very accurate object segmentations is an elaborate task that requires a high degree of quality control and a budget of tens or hundreds of thousands of dollars.
Latent Structured Active Learning
In this paper we present active learning algorithms in the context of structured prediction problems.
Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs
Recent trends in semantic image segmentation have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning.
Bottom-Up Segmentation for Top-Down Detection
When employing the parts, we outperform the original DPM [14] in 19 out of 20 classes, achieving an improvement of 8% AP.
Robust Monocular Epipolar Flow Estimation
We consider the problem of computing optical flow in monocular video taken from a moving vehicle.
A Sentence Is Worth a Thousand Pixels
We are interested in holistic scene understanding where images are accompanied with text in the form of complex sentential descriptions.
Lost! Leveraging the Crowd for Probabilistic Visual Self-Localization
In this paper we propose an affordable solution to selflocalization, which utilizes visual odometry and road maps as the only inputs.
Globally Convergent Dual MAP LP Relaxation Solvers using Fenchel-Young Margins
While finding the exact solution for the MAP inference problem is intractable for many real-world tasks, MAP LP relaxations have been shown to be very effective in practice.
3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model
We demonstrate the effectiveness of our approach in indoor and outdoor scenarios, and show that our approach outperforms the state-of-the-art in both 2D[Felz09] and 3D object detection[Hedau12].
Blending Learning and Inference in Structured Prediction
In this paper we derive an efficient algorithm to learn the parameters of structured predictors in general graphical models.
Learning Probabilistic Non-Linear Latent Variable Models for Tracking Complex Activities
A common approach for handling the complexity and inherent ambiguities of 3D human pose estimation is to use pose priors learned from training data.
