1 code implementation • 23 Feb 2023 • Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M. Alvarez, Sanja Fidler, Chen Feng, Anima Anandkumar
To enable such capability in AI systems, we propose VoxFormer, a Transformer-based semantic scene completion framework that can output complete 3D volumetric semantics from only 2D images.
no code implementations • 9 Feb 2023 • Viraj Prabhu, David Acuna, Andrew Liao, Rafid Mahmood, Marc T. Law, Judy Hoffman, Sanja Fidler, James Lucas
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domain.
no code implementations • 2 Feb 2023 • Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, Xue Bin Peng
These scene interactions are learned using an adversarial discriminator that evaluates the realism of a motion within the context of a scene.
1 code implementation • 31 Jan 2023 • Jordan Juravsky, Yunrong Guo, Sanja Fidler, Xue Bin Peng
In this work, we present PADL, which leverages recent innovations in NLP in order to take steps towards developing language-directed controllers for physics-based character animation.
no code implementations • 18 Nov 2022 • Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, Tsung-Yi Lin
DreamFusion has recently demonstrated the utility of a pre-trained text-to-image diffusion model to optimize Neural Radiance Fields (NeRF), achieving remarkable text-to-3D synthesis results.
1 code implementation • 12 Oct 2022 • Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis
To advance 3D DDMs and make them useful for digital artists, we require (i) high generation quality, (ii) flexibility for manipulation and applications such as conditional synthesis and shape interpolation, and (iii) the ability to output smooth surfaces or meshes.
Ranked #1 on
Point Cloud Generation
on ShapeNet
no code implementations • 6 Oct 2022 • Hassan Abu Alhaija, Alara Dirik, André Knörig, Sanja Fidler, Maria Shugrina
Specifically, we propose a novel method to convert 3D shapes into compact 1-channel geometry images and leverage StyleGAN3 and image-to-image translation networks to generate 3D objects in 2D space.
no code implementations • 3 Oct 2022 • Rafid Mahmood, James Lucas, Jose M. Alvarez, Sanja Fidler, Marc T. Law
Modern deep learning systems require huge data sets to achieve impressive performance, but there is little guidance on how much or what kind of data to collect.
3 code implementations • 26 Sep 2022 • Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard Higgins, Sanja Fidler, David Fouhey, Dima Damen
VISOR annotates videos from EPIC-KITCHENS, which comes with a new set of challenges not encountered in current video segmentation datasets.
2 code implementations • 22 Sep 2022 • Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, Sanja Fidler
As several industries are moving towards modeling massive 3D virtual worlds, the need for content creation tools that can scale in terms of the quantity, quality, and diversity of 3D content is becoming evident.
no code implementations • 19 Aug 2022 • Zian Wang, Wenzheng Chen, David Acuna, Jan Kautz, Sanja Fidler
In this work, we propose a neural approach that estimates the 5D HDR light field from a single image, and a differentiable object insertion formulation that enables end-to-end training with image-based losses that encourage realism.
1 code implementation • 18 Aug 2022 • Gopal Sharma, Kangxue Yin, Subhransu Maji, Evangelos Kalogerakis, Or Litany, Sanja Fidler
As a result, the learned 2D representations are view-invariant and geometrically consistent, leading to better generalization when trained on a limited number of labeled shapes compared to alternatives that utilize self-supervision in 2D or 3D alone.
no code implementations • 5 Jul 2022 • Gary Leung, Jun Gao, Xiaohui Zeng, Sanja Fidler
HILA extends hierarchical vision transformer architectures by adding local connections between features of higher and lower levels to the backbone encoder.
no code implementations • CVPR 2022 • Rafid Mahmood, James Lucas, David Acuna, Daiqing Li, Jonah Philion, Jose M. Alvarez, Zhiding Yu, Sanja Fidler, Marc T. Law
Given a small training data set and a learning algorithm, how much more data is necessary to reach a target validation or test performance?
no code implementations • NeurIPS 2021 • Tianshi Cao, Sasha Doubov, David Acuna, Sanja Fidler
NDS uses a mixture of experts trained on data sources to estimate similarity between each source and the downstream task.
1 code implementation • 15 Jun 2022 • Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, Sanja Fidler
Neural approximations of scalar and vector fields, such as signed distance functions and radiance fields, have emerged as accurate, high-quality representations.
no code implementations • CVPR 2022 • Seung Wook Kim, Karsten Kreis, Daiqing Li, Antonio Torralba, Sanja Fidler
Modern image generative models show remarkable sample quality when trained on a single domain or class of objects.
no code implementations • 4 May 2022 • Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, Sanja Fidler
By leveraging a massively parallel GPU-based simulator, we are able to train skill embeddings using over a decade of simulated experiences, enabling our model to learn a rich and versatile repertoire of skills.
no code implementations • 11 Apr 2022 • Enze Xie, Zhiding Yu, Daquan Zhou, Jonah Philion, Anima Anandkumar, Sanja Fidler, Ping Luo, Jose M. Alvarez
In this paper, we propose M$^2$BEV, a unified framework that jointly performs 3D object detection and map segmentation in the Birds Eye View~(BEV) space with multi-camera image inputs.
no code implementations • CVPR 2022 • Zhiqin Chen, Kangxue Yin, Sanja Fidler
In this paper, we address the problem of texture representation for 3D shapes for the challenging and underexplored tasks of texture transfer and synthesis.
no code implementations • 16 Feb 2022 • Hsueh-Ti Derek Liu, Francis Williams, Alec Jacobson, Sanja Fidler, Or Litany
The latent descriptor of a neural field acts as a deformation handle for the 3D shape it represents.
no code implementations • ICLR 2022 • David Acuna, Marc T Law, Guojun Zhang, Sanja Fidler
Defining optimal solutions in domain-adversarial training as a local Nash equilibrium, we show that gradient descent in domain-adversarial training can violate the asymptotic convergence guarantees of the optimizer, oftentimes hindering the transfer performance.
no code implementations • 8 Feb 2022 • Cinjon Resnick, Or Litany, Amlan Kar, Karsten Kreis, James Lucas, Kyunghyun Cho, Sanja Fidler
Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes.
no code implementations • 20 Jan 2022 • Or Litany, Haggai Maron, David Acuna, Jan Kautz, Gal Chechik, Sanja Fidler
Standard Federated Learning (FL) techniques are limited to clients with identical network architectures.
no code implementations • CVPR 2022 • Daiqing Li, Huan Ling, Seung Wook Kim, Karsten Kreis, Adela Barriuso, Sanja Fidler, Antonio Torralba
By training an effective feature segmentation architecture on top of BigGAN, we turn BigGAN into a labeled dataset generator.
no code implementations • CVPR 2022 • Davis Rempe, Jonah Philion, Leonidas J. Guibas, Sanja Fidler, Or Litany
Scenario generation is formulated as an optimization in the latent space of this traffic model, perturbing an initial real-world scene to produce trajectories that collide with a given planner.
no code implementations • CVPR 2022 • Matan Atzmon, Koki Nagano, Sanja Fidler, Sameh Khamis, Yaron Lipman
A natural way to incorporate symmetries in shape space learning is to ask that the mapping to the shape space (encoder) and mapping from the shape space (decoder) are equivariant to the relevant symmetries.
no code implementations • 2 Dec 2021 • Sourav Biswas, Kangxue Yin, Maria Shugrina, Sanja Fidler, Sameh Khamis
We present HIPNet, a neural implicit pose network trained on multiple subjects across many poses.
no code implementations • NeurIPS 2021 • Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, Karsten Kreis
Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead.
no code implementations • CVPR 2022 • Francis Williams, Zan Gojcic, Sameh Khamis, Denis Zorin, Joan Bruna, Sanja Fidler, Or Litany
We present Neural Kernel Fields: a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression.
2 code implementations • CVPR 2022 • Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, Sanja Fidler
We present an efficient method for joint optimization of topology, materials and lighting from multi-view image observations.
no code implementations • NeurIPS 2021 • David Acuna, Jonah Philion, Sanja Fidler
Alternative solutions seek to exploit driving simulators that can generate large amounts of labeled data with a plethora of content variations.
no code implementations • NeurIPS 2021 • Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, Sanja Fidler
The core of DMTet includes a deformable tetrahedral grid that encodes a discretized signed distance function and a differentiable marching tetrahedra layer that converts the implicit signed distance representation to the explicit surface mesh representation.
1 code implementation • NeurIPS 2021 • Huan Ling, Karsten Kreis, Daiqing Li, Seung Wook Kim, Antonio Torralba, Sanja Fidler
EditGAN builds on a GAN framework that jointly models images and their semantic segmentations, requiring only a handful of labeled examples, making it a scalable tool for editing.
1 code implementation • 1 Nov 2021 • Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, Karsten Kreis
Generative models trained with privacy constraints on private data can sidestep this challenge, providing indirect access to private data instead.
no code implementations • NeurIPS 2021 • Wenzheng Chen, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khamis, Or Litany, Sanja Fidler
We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiable renderers.
1 code implementation • NeurIPS 2021 • Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, Sanja Fidler
The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many applications, from better interactive 3D tools to data synthesis for training and simulation.
no code implementations • ICLR 2022 • Rafid Mahmood, Sanja Fidler, Marc T Law
Active learning is the process of training a model with limited labeled data by selecting a core subset of an unlabeled data pool to label.
no code implementations • 29 Sep 2021 • Cinjon Resnick, Or Litany, Amlan Kar, Karsten Kreis, James Lucas, Kyunghyun Cho, Sanja Fidler
We verify that the prioritized groups found via intervention are challenging for the object detector and show that retraining with data collected from these groups helps inordinately compared to adding more IID data.
no code implementations • ICCV 2021 • Kevin Xie, Tingwu Wang, Umar Iqbal, Yunrong Guo, Sanja Fidler, Florian Shkurti
By enabling learning of motion synthesis from video, our method paves the way for large-scale, realistic and diverse motion synthesis.
no code implementations • ICCV 2021 • Zian Wang, Jonah Philion, Sanja Fidler, Jan Kautz
In this paper, we propose a unified, learning-based inverse rendering framework that formulates 3D spatially-varying lighting.
no code implementations • ICCV 2021 • Kangxue Yin, Jun Gao, Maria Shugrina, Sameh Khamis, Sanja Fidler
Given a small set of high-quality textured objects, our method can create many novel stylized shapes, resulting in effortless 3D content creation and style-ware data augmentation.
1 code implementation • 25 Jun 2021 • Xiaohui Zeng, Raquel Urtasun, Richard Zemel, Sanja Fidler, Renjie Liao
1) We propose a non-parametric prior distribution over the appearance of image parts so that the latent variable ``what-to-draw'' per step becomes a categorical random variable.
1 code implementation • 21 Jun 2021 • David Acuna, Guojun Zhang, Marc T. Law, Sanja Fidler
Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain, and a related labeled dataset.
no code implementations • 5 Jun 2021 • Rafid Mahmood, Sanja Fidler, Marc T. Law
Active learning is the process of training a model with limited labeled data by selecting a core subset of an unlabeled data pool to label.
no code implementations • CVPR 2021 • Seung Wook Kim, Jonah Philion, Antonio Torralba, Sanja Fidler
Realistic simulators are critical for training and verifying robotics systems.
1 code implementation • CVPR 2021 • Yuan-Hong Liao, Amlan Kar, Sanja Fidler
This is expensive, and guaranteeing the quality of the labels is a major challenge.
1 code implementation • CVPR 2021 • Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Jean-Francois Lafleche, Adela Barriuso, Antonio Torralba, Sanja Fidler
To showcase the power of our approach, we generated datasets for 7 image segmentation tasks which include pixel-level labels for 34 human face parts, and 32 car parts.
no code implementations • CVPR 2021 • Daiqing Li, Junlin Yang, Karsten Kreis, Antonio Torralba, Sanja Fidler
Training deep networks with limited labeled data while achieving a strong generalization ability is key in the quest to reduce human annotation efforts.
1 code implementation • 12 Apr 2021 • Nadine Chang, Zhiding Yu, Yu-Xiong Wang, Anima Anandkumar, Sanja Fidler, Jose M. Alvarez
As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level.
no code implementations • ICLR 2021 • Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jerome Parent-Levesque, Kevin Xie, Kenny Erleben, Liam Paull, Florian Shkurti, Derek Nowrouzezahrai, Sanja Fidler
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences.
1 code implementation • CVPR 2021 • Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, Sanja Fidler
The INN allows us to compute the inverse mapping of the homeomorphism, which in turn, enables the efficient computation of both the implicit surface function of a primitive and its mesh, without any additional post-processing.
2 code implementations • CVPR 2021 • Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, Sanja Fidler
We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs, while achieving state-of-the-art geometry reconstruction quality.
no code implementations • 1 Jan 2021 • David Acuna, Guojun Zhang, Marc T Law, Sanja Fidler
We provide empirical results for several f-divergences and show that some, not considered previously in domain-adversarial learning, achieve state-of-the-art results in practice.
no code implementations • 1 Jan 2021 • Tianshi Cao, Alex Bie, Karsten Kreis, Sanja Fidler
Generative models trained with privacy constraints on private data can sidestep this challenge and provide indirect access to the private data instead.
3 code implementations • ICLR 2021 • Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, Jose M. Alvarez
While federated learning traditionally aims to train a single global model across decentralized local datasets, one model may not always be ideal for all participating clients.
no code implementations • NeurIPS 2020 • Huan Ling, David Acuna, Karsten Kreis, Seung Wook Kim, Sanja Fidler
In images of complex scenes, objects are often occluding each other which makes perception tasks such as object detection and tracking, or robotic control tasks such as planning, challenging.
no code implementations • 30 Nov 2020 • Tingwu Wang, Yunrong Guo, Maria Shugrina, Sanja Fidler
The field of physics-based animation is gaining importance due to the increasing demand for realism in video games and films, and has recently seen wide adoption of data-driven techniques, such as deep reinforcement learning (RL), which learn control from (human) demonstrations.
1 code implementation • ICLR 2021 • Avik Pal, Jonah Philion, Yuan-Hong Liao, Sanja Fidler
For autonomous vehicles to safely share the road with human drivers, autonomous vehicles must abide by specific "road rules" that human drivers have agreed to follow.
1 code implementation • NeurIPS 2020 • Jun Gao, Wenzheng Chen, Tommy Xiang, Clement Fuji Tsang, Alec Jacobson, Morgan McGuire, Sanja Fidler
We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
1 code implementation • ICLR 2021 • Xavier Puig, Tianmin Shu, Shuang Li, Zilin Wang, Yuan-Hong Liao, Joshua B. Tenenbaum, Sanja Fidler, Antonio Torralba
In this paper, we introduce Watch-And-Help (WAH), a challenge for testing social intelligence in agents.
1 code implementation • 19 Oct 2020 • Yiluan Guo, Holger Caesar, Oscar Beijbom, Jonah Philion, Sanja Fidler
A high-performing object detection system plays a crucial role in autonomous driving (AD).
no code implementations • ICLR 2021 • Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler
Key to our approach is to exploit GANs as a multi-view data generator to train an inverse graphics network using an off-the-shelf differentiable renderer, and the trained inverse graphics network as a teacher to disentangle the GAN's latent code into interpretable 3D properties.
no code implementations • 1 Sep 2020 • Daiqing Li, Amlan Kar, Nishant Ravikumar, Alejandro F. Frangi, Sanja Fidler
Since the model of geometry and material is disentangled from the imaging sensor, it can effectively be trained across multiple medical centers.
no code implementations • ECCV 2020 • Hang Chu, Shugao Ma, Fernando de la Torre, Sanja Fidler, Yaser Sheikh
It is important to note that traditional person-specific CAs are learned from few training samples, and typically lack robustness as well as limited expressiveness when transferring facial expressions.
no code implementations • ECCV 2020 • Tianchang Shen, Jun Gao, Amlan Kar, Sanja Fidler
We implement our framework as a web service and conduct a user study, where we show that user annotated data using our method effectively facilitates real-world learning tasks.
no code implementations • ECCV 2020 • Bo-Wen Chen, Huan Ling, Xiaohui Zeng, Gao Jun, Ziyue Xu, Sanja Fidler
Our approach tolerates a modest amount of noise in the box placements, thus typically only a few clicks are needed to annotate tracked boxes to a sufficient accuracy.
no code implementations • ECCV 2020 • Jun Gao, Zian Wang, Jinchen Xuan, Sanja Fidler
We also utilize DefGrid at the output layers for the task of object mask annotation, and show that reasoning about object boundaries on our predicted polygonal grid leads to more accurate results over existing pixel-wise and curve-based approaches.
no code implementations • ECCV 2020 • Jeevan Devaranjan, Amlan Kar, Sanja Fidler
In Meta-Sim2, we aim to learn the scene structure in addition to parameters, which is a challenging problem due to its discrete nature.
no code implementations • 18 Aug 2020 • Jiaman Li, Yihang Yin, Hang Chu, Yi Zhou, Tingwu Wang, Sanja Fidler, Hao Li
We also introduce new evaluation metrics for the quality of synthesized dance motions, and demonstrate that our system can outperform state-of-the-art methods.
1 code implementation • ECCV 2020 • Jonah Philion, Sanja Fidler
By training on the entire camera rig, we provide evidence that our model is able to learn not only how to represent images but how to fuse predictions from all cameras into a single cohesive representation of the scene while being robust to calibration error.
no code implementations • CVPR 2020 • Seung Wook Kim, Yuhao Zhou, Jonah Philion, Antonio Torralba, Sanja Fidler
Simulation is a crucial component of any robotic system.
1 code implementation • ICLR 2020 • Wei Yu, Yichao Lu, Steve Easterbrook, Sanja Fidler
Applying resolution-preserving blocks is a common practice to maximize information preservation in video prediction, yet their high memory consumption greatly limits their application scenarios.
Ranked #13 on
Video Prediction
on Moving MNIST
2 code implementations • 29 Apr 2020 • Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray
Our dataset features 55 hours of video consisting of 11. 5M frames, which we densely labelled for a total of 39. 6K action segments and 454. 2K object bounding boxes.
no code implementations • CVPR 2020 • Jonah Philion, Amlan Kar, Sanja Fidler
The downside of these metrics is that, at worst, they penalize all incorrect detections equally without conditioning on the task or scene, and at best, heuristics need to be chosen to ensure that different mistakes count differently.
no code implementations • CVPR 2020 • Xi Yan, David Acuna, Sanja Fidler
NDS consists of a dataserver which indexes several large popular image datasets, and aims to recommend data to a client, an end-user with a target application with its own small labeled dataset.
1 code implementation • 30 Dec 2019 • Atef Chaudhury, Makarand Tapaswi, Seung Wook Kim, Sanja Fidler
Understanding stories is a challenging reading comprehension problem for machines as it requires reading a large volume of text and following long-range dependencies.
6 code implementations • 12 Nov 2019 • Krishna Murthy Jatavallabhula, Edward Smith, Jean-Francois Lafleche, Clement Fuji Tsang, Artem Rozantsev, Wenzheng Chen, Tommy Xiang, Rev Lebaredian, Sanja Fidler
We present Kaolin, a PyTorch library aiming to accelerate 3D deep learning research.
no code implementations • 25 Oct 2019 • Wei Yu, Yichao Lu, Steve Easterbrook, Sanja Fidler
Applying resolution-preserving blocks is a common practice to maximize information preservation in video prediction, yet their high memory consumption greatly limits their application scenarios.
no code implementations • ICCV 2019 • Hang Chu, Daiqing Li, David Acuna, Amlan Kar, Maria Shugrina, Xinkai Wei, Ming-Yu Liu, Antonio Torralba, Sanja Fidler
We propose Neural Turtle Graphics (NTG), a novel generative model for spatial graphs, and demonstrate its applications in modeling city road layouts.
1 code implementation • ICCV 2019 • Xiaohui Zeng, Renjie Liao, Li Gu, Yuwen Xiong, Sanja Fidler, Raquel Urtasun
In practice, it performs similarly to the Hungarian algorithm during inference.
One-shot visual object segmentation
Semantic Segmentation
+1
no code implementations • ICLR 2020 • Tianshi Cao, Marc Law, Sanja Fidler
We introduce a theoretical analysis of the impact of the shot number on Prototypical Networks, a state-of-the-art few-shot classification method.
1 code implementation • ICCV 2019 • Makarand Tapaswi, Marc T. Law, Sanja Fidler
Understanding videos such as TV series and movies requires analyzing who the characters are and what they are doing.
1 code implementation • NeurIPS 2019 • Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler
Many machine learning models operate on images, but ignore the fact that images are 2D projections formed by 3D geometry interacting with light, in a process called rendering.
Ranked #4 on
Single-View 3D Reconstruction
on ShapeNet
4 code implementations • ICCV 2019 • Towaki Takikawa, David Acuna, Varun Jampani, Sanja Fidler
Here, we propose a new two-stream CNN architecture for semantic segmentation that explicitly wires shape information as a separate processing branch, i. e. shape stream, that processes information in parallel to the classical stream.
Ranked #18 on
Semantic Segmentation
on Cityscapes test
1 code implementation • 12 Jun 2019 • Tingwu Wang, Yuhao Zhou, Sanja Fidler, Jimmy Ba
To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space.
1 code implementation • 15 May 2019 • Chaoqi Wang, Roger Grosse, Sanja Fidler, Guodong Zhang
Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices.
no code implementations • CVPR 2019 • Dominic Cheng, Renjie Liao, Sanja Fidler, Raquel Urtasun
In this paper, we propose a Deep Active Ray Network (DARNet) for automatic building segmentation.
no code implementations • ICLR 2019 • Tingwu Wang, Yuhao Zhou, Sanja Fidler, Jimmy Ba
To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space.
no code implementations • ICLR 2019 • Yuhuai Wu, Harris Chan, Jamie Kiros, Sanja Fidler, Jimmy Ba
Sparse reward is one of the most challenging problems in reinforcement learning (RL).
no code implementations • ICCV 2019 • Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler
Training models to high-end performance requires availability of large labeled datasets, which are expensive to get.
1 code implementation • CVPR 2019 • David Acuna, Amlan Kar, Sanja Fidler
We further reason about true object boundaries during training using a level set formulation, which allows the network to learn from misaligned labels in an end-to-end fashion.
1 code implementation • CVPR 2019 • Davide Moltisanti, Sanja Fidler, Dima Damen
We propose a method that is supervised by single timestamps located around each action instance, in untrimmed videos.
no code implementations • 27 Mar 2019 • Jun Gao, Xiao Li, Li-Wei Wang, Sanja Fidler, Stephen Lin
We present a method for compositing virtual objects into a photograph such that the object colors appear to have been processed by the photo's camera imaging pipeline.
2 code implementations • CVPR 2019 • Huan Ling, Jun Gao, Amlan Kar, Wenzheng Chen, Sanja Fidler
Our model runs at 29. 3ms in automatic, and 2. 6ms in interactive mode, making it 10x and 100x faster than Polygon-RNN++.
no code implementations • 12 Feb 2019 • Harris Chan, Yuhuai Wu, Jamie Kiros, Sanja Fidler, Jimmy Ba
We first analyze the differences among goal representation, and show that ACTRCE can efficiently solve difficult reinforcement learning problems in challenging 3D navigation tasks, whereas HER with non-language goal representation failed to learn.
1 code implementation • CVPR 2018 • Hang Chu, Wei-Chiu Ma, Kaustav Kundu, Raquel Urtasun, Sanja Fidler
On the other hand, 3D convolution wastes a large amount of memory on mostly unoccupied 3D space, which consists of only the surface visible to the sensor.
no code implementations • CVPR 2018 • Hang Chu, Daiqing Li, Sanja Fidler
The decoder consists of two layers, where the lower layer aims at generating the verbal response and coarse facial expressions, while the second layer fills in the subtle gestures, making the generated output more smooth and natural.
1 code implementation • 1 Dec 2018 • Kevin Shen, Amlan Kar, Sanja Fidler
In order to bring artificial agents into our lives, we will need to go beyond supervised learning on closed datasets to having the ability to continuously expand knowledge.
1 code implementation • NeurIPS 2018 • Bo Dai, Sanja Fidler, Dahua Lin
Mainstream captioning models often follow a sequential structure to generate captions, leading to issues such as introduction of irrelevant semantics, lack of diversity in the generated captions, and inadequate generalization performance.
no code implementations • 13 Oct 2018 • Enric Corona, Kaustav Kundu, Sanja Fidler
In particular, our aim is to infer poses for objects not seen at training time, but for which their 3D CAD models are available at test time.
4 code implementations • CVPR 2018 • Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, Antonio Torralba
We then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment.
no code implementations • 7 Jun 2018 • Maria Shugrina, Amlan Kar, Karan Singh, Sanja Fidler
Then, the user can adjust color sail parameters to change the base colors, their blending behavior and the number of colors, exploring a wide range of options for the original design.
1 code implementation • ICLR 2019 • Seung Wook Kim, Makarand Tapaswi, Sanja Fidler
Thus, a module for a new task learns to query existing modules and composes their outputs in order to produce its own output.
no code implementations • CVPR 2018 • Yuhao Zhou, Makarand Tapaswi, Sanja Fidler
We are interested in enabling automatic 4D cinema by parsing physical and special effects from untrimmed movies.
2 code implementations • ECCV 2018 • Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, Michael Wray
First-person vision is gaining interest as it offers a unique viewpoint on people's interaction with objects, their attention, and even intention.
3 code implementations • CVPR 2018 • David Acuna, Huan Ling, Amlan Kar, Sanja Fidler
Manually labeling datasets with object masks is extremely time consuming.
1 code implementation • ICLR 2018 • Tingwu Wang, Renjie Liao, Jimmy Ba, Sanja Fidler
We address the problem of learning structured policies for continuous control.
no code implementations • CVPR 2018 • Ching-Yao Chuang, Jiaman Li, Antonio Torralba, Sanja Fidler
We address the problem of affordance reasoning in diverse scenes that appear in the real world.
no code implementations • CVPR 2018 • Paul Vicol, Makarand Tapaswi, Lluis Castrejon, Sanja Fidler
Towards this goal, we introduce a novel dataset called MovieGraphs which provides detailed, graph-based annotations of social situations depicted in movie clips.
no code implementations • NeurIPS 2017 • Huan Ling, Sanja Fidler
Robots will eventually be part of every household.
no code implementations • ICCV 2017 • Shizhan Zhu, Sanja Fidler, Raquel Urtasun, Dahua Lin, Chen Change Loy
In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.
no code implementations • ICCV 2017 • Shu Liu, Jiaya Jia, Sanja Fidler, Raquel Urtasun
By exploiting two-directional information, the second network groups horizontal and vertical lines into connected components.
2 code implementations • ICCV 2017 • Xiaojuan Qi, Renjie Liao, Jiaya Jia, Sanja Fidler, Raquel Urtasun
Each node in the graph corresponds to a set of points and is associated with a hidden representation vector initialized with an appearance feature extracted by a unary CNN from 2D images.
Ranked #21 on
Semantic Segmentation
on SUN-RGBD
1 code implementation • ICCV 2017 • Ruiyu Li, Makarand Tapaswi, Renjie Liao, Jiaya Jia, Raquel Urtasun, Sanja Fidler
We address the problem of recognizing situations in images.
Ranked #8 on
Grounded Situation Recognition
on SWiG
9 code implementations • 18 Jul 2017 • Fartash Faghri, David J. Fleet, Jamie Ryan Kiros, Sanja Fidler
We present a new technique for learning visual-semantic embeddings for cross-modal retrieval.
Ranked #19 on
Cross-Modal Retrieval
on Flickr30k
no code implementations • CVPR 2017 • Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, Antonio Torralba
A novel network design called Cascade Segmentation Module is proposed to parse a scene into stuff, objects, and object parts in a cascade and improve over the baselines.
no code implementations • CVPR 2017 • Namdar Homayounfar, Sanja Fidler, Raquel Urtasun
In this work, we propose a novel way of efficiently localizing a sports field from a single broadcast image of the game.
no code implementations • 1 Jun 2017 • Huan Ling, Sanja Fidler
Robots will eventually be part of every household.
2 code implementations • CVPR 2017 • Lluis Castrejon, Kaustav Kundu, Raquel Urtasun, Sanja Fidler
We show that our approach speeds up the annotation process by a factor of 4. 7 across all classes in Cityscapes, while achieving 78. 4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators.
no code implementations • ICCV 2017 • Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler, Antonio Torralba
Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets.
1 code implementation • ICCV 2017 • Bo Dai, Sanja Fidler, Raquel Urtasun, Dahua Lin
Despite the substantial progress in recent years, the image captioning techniques are still far from being perfect. Sentences produced by existing methods, e. g. those based on RNNs, are often overly rigid and lacking in variability.
no code implementations • ICCV 2017 • Shenlong Wang, Min Bai, Gellert Mattyus, Hang Chu, Wenjie Luo, Bin Yang, Justin Liang, Joel Cheverie, Sanja Fidler, Raquel Urtasun
In this paper we introduce the TorontoCity benchmark, which covers the full greater Toronto area (GTA) with 712. 5 $km^2$ of land, 8439 $km$ of road and around 400, 000 buildings.
no code implementations • NeurIPS 2016 • Shenlong Wang, Sanja Fidler, Raquel Urtasun
Many problems in real-world applications involve predicting continuous-valued random variables that are statistically related.
no code implementations • 10 Nov 2016 • Hang Chu, Raquel Urtasun, Sanja Fidler
We present a novel framework for generating pop music.
no code implementations • 10 Nov 2016 • Wenyuan Zeng, Wenjie Luo, Sanja Fidler, Raquel Urtasun
Towards this goal, we first introduce a simple mechanism that first reads the input sequence before committing to a representation of each word.
no code implementations • 27 Aug 2016 • Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Huimin Ma, Sanja Fidler, Raquel Urtasun
We then exploit a CNN on top of these proposals to perform object detection.
21 code implementations • 18 Aug 2016 • Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, Antonio Torralba
Scene parsing, or recognizing and segmenting objects and stuff in an image, is one of the key problems in computer vision.
no code implementations • 23 Jun 2016 • Wei-Chiu Ma, Shenlong Wang, Marcus A. Brubaker, Sanja Fidler, Raquel Urtasun
In this paper we present a robust, efficient and affordable approach to self-localization which does not require neither GPS nor knowledge about the appearance of the world.
no code implementations • CVPR 2016 • Gellert Mattyus, Shenlong Wang, Sanja Fidler, Raquel Urtasun
In this paper we present an approach to enhance existing maps with fine grained segmentation categories such as parking spots and sidewalk, as well as the number and location of road lanes.
no code implementations • CVPR 2016 • Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, Raquel Urtasun
The focus of this paper is on proposal generation.
Ranked #8 on
Vehicle Pose Estimation
on KITTI Cars Hard
no code implementations • 10 Apr 2016 • Namdar Homayounfar, Sanja Fidler, Raquel Urtasun
In this work, we propose a novel way of efficiently localizing a soccer field from a single broadcast image of the game.
no code implementations • CVPR 2016 • Ziyu Zhang, Sanja Fidler, Raquel Urtasun
Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving.
1 code implementation • CVPR 2016 • Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, Sanja Fidler
We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text.
no code implementations • NeurIPS 2015 • Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Andrew G. Berneshawi, Huimin Ma, Sanja Fidler, Raquel Urtasun
The goal of this paper is to generate high-quality 3D object proposals in the context of autonomous driving.
Ranked #10 on
Vehicle Pose Estimation
on KITTI Cars Hard
no code implementations • ICCV 2015 • Shenlong Wang, Sanja Fidler, Raquel Urtasun
In this paper we propose a novel approach to localization in very large indoor spaces (i. e., 200+ store shopping malls) that takes a single image and a floor plan of the environment as input.
no code implementations • ICCV 2015 • Tom Lee, Sanja Fidler, Sven Dickinson
In this paper, we introduce Parametric Min-Loss (PML), a novel structured learning framework for parametric energy functions.
no code implementations • ICCV 2015 • Gellert Mattyus, Shenlong Wang, Sanja Fidler, Raquel Urtasun
In recent years, contextual models that exploit maps have been shown to be very effective for many recognition and localization tasks.
2 code implementations • 19 Nov 2015 • Ivan Vendrov, Ryan Kiros, Sanja Fidler, Raquel Urtasun
Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images.
Ranked #89 on
Natural Language Inference
on SNLI
16 code implementations • NeurIPS 2015 • Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, Sanja Fidler
The end result is an off-the-shelf encoder that can produce highly generic sentence representations that are robust and perform well in practice.
Ranked #2 on
Semantic Similarity
on SICK
3 code implementations • ICCV 2015 • Yukun Zhu, Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, Sanja Fidler
Books are a rich source of both fine-grained information, how a character, an object or a scene looks like, as well as high-level semantics, what someone is thinking, feeling and how these states evolve through a story.
no code implementations • ICCV 2015 • Jimmy Ba, Kevin Swersky, Sanja Fidler, Ruslan Salakhutdinov
One of the main challenges in Zero-Shot Learning of visual categories is gathering semantic attributes to accompany images.
no code implementations • CVPR 2015 • Shenlong Wang, Sanja Fidler, Raquel Urtasun
In this paper we are interested in exploiting geographic priors to help outdoor scene understanding.
no code implementations • CVPR 2015 • Chenxi Liu, Alexander G. Schwing, Kaustav Kundu, Raquel Urtasun, Sanja Fidler
What sets us apart from past work in layout estimation is the use of floor plans as a source of prior knowledge, as well as localization of each image within a bigger space (apartment).
no code implementations • CVPR 2015 • Jian Yao, Marko Boben, Sanja Fidler, Raquel Urtasun
In this paper, we tackle the problem of unsupervised segmentation in the form of superpixels.
no code implementations • ICCV 2015 • Ziyu Zhang, Alexander G. Schwing, Sanja Fidler, Raquel Urtasun
In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image.
no code implementations • 28 Feb 2015 • Dahua Lin, Chen Kong, Sanja Fidler, Raquel Urtasun
This paper proposes a novel framework for generating lingual descriptions of indoor scenes.
no code implementations • CVPR 2015 • Yukun Zhu, Raquel Urtasun, Ruslan Salakhutdinov, Sanja Fidler
In this paper, we propose an approach that exploits object segmentation in order to improve the accuracy of object detection.
no code implementations • 5 Feb 2015 • Tom Lee, Sanja Fidler, Alex Levinshtein, Cristian Sminchisescu, Sven Dickinson
The role of symmetry in computer vision has waxed and waned in importance during the evolution of the field from its earliest days.
no code implementations • Conference 2015 • Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, Raquel Urtasun
Importantly, our model is able to give rich feedback back to the user, conveying which garments or even scenery she/he should change in order to improve fashionability.
no code implementations • 23 Aug 2014 • Sanja Fidler, Marko Boben, Ales Leonardis
At the top-level of the vocabulary, the compositions are sufficiently large and complex to represent the whole shapes of the objects.
no code implementations • 9 Aug 2014 • Andrei Barbu, Alexander Bridge, Zachary Burchill, Dan Coroian, Sven Dickinson, Sanja Fidler, Aaron Michaux, Sam Mussman, Siddharth Narayanaswamy, Dhaval Salvi, Lara Schmidt, Jiangnan Shangguan, Jeffrey Mark Siskind, Jarrell Waggoner, Song Wang, Jinlian Wei, Yifan Yin, Zhiqi Zhang
We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it.
no code implementations • 16 Jun 2014 • Roozbeh Mottaghi, Sanja Fidler, Alan Yuille, Raquel Urtasun, Devi Parikh
Recent trends in image understanding have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning, and local appearance based classifiers.
no code implementations • CVPR 2014 • Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fidler, Raquel Urtasun, Alan Yuille
Our model automatically decouples the holistic object or body parts from the model when they are hard to detect.
no code implementations • CVPR 2014 • Chen Kong, Dahua Lin, Mohit Bansal, Raquel Urtasun, Sanja Fidler
In this paper we exploit natural sentential descriptions of RGB-D scenes in order to improve 3D semantic parsing.
1 code implementation • CVPR 2014 • Liang-Chieh Chen, Sanja Fidler, Alan L. Yuille, Raquel Urtasun
Labeling large-scale datasets with very accurate object segmentations is an elaborate task that requires a high degree of quality control and a budget of tens or hundreds of thousands of dollars.
no code implementations • CVPR 2014 • Dahua Lin, Sanja Fidler, Chen Kong, Raquel Urtasun
In this paper, we tackle the problem of retrieving videos using complex natural language queries.
no code implementations • CVPR 2014 • Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, Alan Yuille
In this paper we study the role of context in existing state-of-the-art detection and segmentation approaches.
no code implementations • CVPR 2013 • Sanja Fidler, Roozbeh Mottaghi, Alan Yuille, Raquel Urtasun
When employing the parts, we outperform the original DPM [14] in 19 out of 20 classes, achieving an improvement of 8% AP.
no code implementations • CVPR 2013 • Roozbeh Mottaghi, Sanja Fidler, Jian Yao, Raquel Urtasun, Devi Parikh
Recent trends in semantic image segmentation have pushed for holistic scene understanding models that jointly reason about various tasks such as object detection, scene recognition, shape analysis, contextual reasoning.
no code implementations • CVPR 2013 • Sanja Fidler, Abhishek Sharma, Raquel Urtasun
We are interested in holistic scene understanding where images are accompanied with text in the form of complex sentential descriptions.
no code implementations • NeurIPS 2012 • Sanja Fidler, Sven Dickinson, Raquel Urtasun
We demonstrate the effectiveness of our approach in indoor and outdoor scenarios, and show that our approach outperforms the state-of-the-art in both 2D[Felz09] and 3D object detection[Hedau12].
no code implementations • NeurIPS 2009 • Sanja Fidler, Marko Boben, Ales Leonardis
We explore and compare their computational behavior (space and time) and detection performance as a function of the number of learned classes on several recognition data sets.