no code implementations • 7 Nov 2024 • Shuhong Zheng, Zhipeng Bao, Ruoyu Zhao, Martial Hebert, Yu-Xiong Wang
Beyond high-fidelity image synthesis, diffusion models have recently exhibited promising results in dense visual perception tasks.
no code implementations • 30 Oct 2024 • Anurag Bagchi, Zhipeng Bao, Yu-Xiong Wang, Pavel Tokmakov, Martial Hebert
We present REM, a framework for segmenting a wide range of concepts in video that can be described through natural language.
1 code implementation • 5 Sep 2024 • Yunze Man, Shuhong Zheng, Zhipeng Bao, Martial Hebert, Liang-Yan Gui, Yu-Xiong Wang
To address this issue, we present a comprehensive study that probes various visual encoding models for 3D scene understanding, identifying the strengths and limitations of each model across different scenarios.
Ranked #3 on
Question Answering
on SQA3D
no code implementations • 10 Dec 2023 • Zhipeng Bao, Yijun Li, Krishna Kumar Singh, Yu-Xiong Wang, Martial Hebert
Despite recent significant strides achieved by diffusion-based Text-to-Image (T2I) models, current systems are still less capable of ensuring decent compositional generation aligned with text prompts, particularly for the multi-object generation.
1 code implementation • ICCV 2023 • Shuhong Zheng, Zhipeng Bao, Martial Hebert, Yu-Xiong Wang
To tackle the MTVS problem, we propose MuvieNeRF, a framework that incorporates both multi-task and cross-view knowledge to simultaneously synthesize multiple scene properties.
no code implementations • 28 Aug 2023 • Leonid Keselman, Martial Hebert
Fast, reliable shape reconstruction is an essential ingredient in many computer vision applications.
no code implementations • 8 Aug 2023 • Leonid Keselman, Katherine Shih, Martial Hebert, Aaron Steinfeld
Typical black-box optimization approaches in robotics focus on learning from metric scores.
no code implementations • 24 Jun 2023 • Nadine Chang, Francesco Ferroni, Michael J. Tarr, Martial Hebert, Deva Ramanan
In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples.
2 code implementations • CVPR 2023 • Zhipeng Bao, Pavel Tokmakov, Yu-Xiong Wang, Adrien Gaidon, Martial Hebert
Object discovery -- separating objects from the background without manual labels -- is a fundamental open challenge in computer vision.
no code implementations • 13 Mar 2023 • Leonid Keselman, Martial Hebert
In this work, we extend algorithm configuration to automatically discover multiple modes in the tuning dataset.
no code implementations • 21 Nov 2022 • Brian Okorn, Chuer Pan, Martial Hebert, David Held
While self-supervised learning has been used successfully for translational object keypoints, in this work, we show that naively applying relative supervision to the rotational group $SO(3)$ will often fail to converge due to the non-convexity of the rotational space.
1 code implementation • 25 Aug 2022 • Bowei Chen, Tiancheng Zhi, Martial Hebert, Srinivasa G. Narasimhan
To address these challenges, we learn a neural implicit representation using a coordinate-based MLP with single image optimization.
1 code implementation • 21 Jul 2022 • Leonid Keselman, Martial Hebert
In this work, we develop an approximate differentiable renderer for a compact, interpretable representation, which we call Fuzzy Metaballs.
no code implementations • 9 Jun 2022 • Mingtong Zhang, Shuhong Zheng, Zhipeng Bao, Martial Hebert, Yu-Xiong Wang
Comprehensive 3D scene understanding, both geometrically and semantically, is important for real-world applications such as robot perception.
1 code implementation • CVPR 2022 • Zhipeng Bao, Pavel Tokmakov, Allan Jabri, Yu-Xiong Wang, Adrien Gaidon, Martial Hebert
Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects.
no code implementations • 29 Sep 2021 • Zhipeng Bao, Yu-Xiong Wang, Martial Hebert
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images.
no code implementations • 25 Jun 2021 • Zhipeng Bao, Martial Hebert, Yu-Xiong Wang
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images.
no code implementations • 20 Jun 2021 • Satyaki Chakraborty, Martial Hebert
Occlusion is one of the most significant challenges encountered by object detectors and trackers.
1 code implementation • 28 Apr 2021 • Brian Okorn, Qiao Gu, Martial Hebert, David Held
We also demonstrate how our system can be used by quickly scanning and building a model of a novel object, which can immediately be used by our method for pose estimation.
no code implementations • ICCV 2021 • Liangke Gui, Adrien Bardes, Ruslan Salakhutdinov, Alexander Hauptmann, Martial Hebert, Yu-Xiong Wang
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks.
no code implementations • 17 Dec 2020 • Xia Chen, Jianren Wang, David Held, Martial Hebert
Visual data in autonomous driving perception, such as camera image and LiDAR point cloud, can be interpreted as a mixture of two aspects: semantic feature and geometric structure.
1 code implementation • 15 Oct 2020 • Zuxin Liu, Hongyi Zhou, Baiming Chen, Sicheng Zhong, Martial Hebert, Ding Zhao
We propose a model-based approach to enable RL agents to effectively explore the environment with unknown system dynamics and environment constraints given a significantly small number of violation budgets.
Model-based Reinforcement Learning
Model Predictive Control
+4
no code implementations • 22 Aug 2020 • Vivek Roy, Yan Xu, Yu-Xiong Wang, Kris Kitani, Ruslan Salakhutdinov, Martial Hebert
Recent works have proposed to solve this task by augmenting the training data of the few-shot classes using generative models with the few-shot training samples as the seeds.
1 code implementation • 17 Aug 2020 • Nadine Chang, Jayanth Koushik, Aarti Singh, Martial Hebert, Yu-Xiong Wang, Michael J. Tarr
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes.
1 code implementation • ICLR 2021 • Zhipeng Bao, Yu-Xiong Wang, Martial Hebert
We propose a novel task of joint few-shot recognition and novel-view synthesis: given only one or few images of a novel object from arbitrary views with only category annotation, we aim to simultaneously learn an object classifier and generate images of that type of object from new viewpoints.
no code implementations • 1 Aug 2020 • Xia Chen, Jianren Wang, Martial Hebert
We propose a simple, fast, and flexible framework to generate simultaneously semantic and instance masks for panoptic segmentation.
no code implementations • 30 Jul 2020 • Zuxin Liu, Baiming Chen, Hongyi Zhou, Guru Koushik, Martial Hebert, Ding Zhao
Multi-agent navigation in dynamic environments is of great industrial value when deploying a large scale fleet of robot to real-world applications.
1 code implementation • 2 Jul 2020 • Brian Okorn, Mengyun Xu, Martial Hebert, David Held
Our first method, which regresses from deep learned features to an isotropic Bingham distribution, gives the best performance for orientation distribution estimation for non-symmetric objects.
1 code implementation • 28 Jun 2020 • Pavel Tokmakov, Martial Hebert, Cordelia Schmid
This paper addresses the task of unsupervised learning of representations for action recognition in videos.
1 code implementation • 29 Nov 2019 • Ziqi Pang, Zhiyuan Hu, Pavel Tokmakov, Yu-Xiong Wang, Martial Hebert
Indeed, even the majority of few-shot learning methods rely on a large set of "base classes" for pretraining.
no code implementations • 15 Oct 2019 • Jean Oh, Martial Hebert, Hae-Gon Jeon, Xavier Perez, Chia Dai, Yeeho Song
One of the key challenges in the semantic mapping problem in postdisaster environments is how to analyze a large amount of data efficiently with minimal supervision.
no code implementations • 25 Sep 2019 • Yu-Xiong Wang, Yuki Uchiyama, Martial Hebert, Karteek Alahari
Learning to hallucinate additional examples has recently been shown as a promising direction to address few-shot learning tasks, which aim to learn novel concepts from very few examples.
1 code implementation • 27 Jul 2019 • Kashyap Chitta, Jose M. Alvarez, Martial Hebert
Semantic segmentation with Convolutional Neural Networks is a memory-intensive task due to the high spatial resolution of feature maps and output predictions.
no code implementations • CVPR 2017 • Yu-Xiong Wang, Deva Ramanan, Martial Hebert
One of their remarkable properties is the ability to transfer knowledge from a large source dataset to a (typically smaller) target dataset.
2 code implementations • 11 Jun 2019 • Kevin Christensen, Martial Hebert
In contrast our method builds on direct visual odometry methods naturally with minimal added computation.
1 code implementation • CVPR 2019 • Zitian Chen, Yanwei Fu, Yu-Xiong Wang, Lin Ma, Wei Liu, Martial Hebert
Humans can robustly learn novel visual concepts even when images undergo various deformations and lose certain information.
1 code implementation • 7 May 2019 • Tejas Khot, Shubham Agrawal, Shubham Tulsiani, Christoph Mertz, Simon Lucey, Martial Hebert
We demonstrate our ability to learn MVS without 3D supervision using a real dataset, and show that each component of our proposed robust loss results in a significant improvement.
no code implementations • 29 Apr 2019 • Yubo Zhang, Pavel Tokmakov, Martial Hebert, Cordelia Schmid
In this work we study the problem of action detection in a highly-imbalanced dataset.
1 code implementation • 11 Apr 2019 • Leonid Keselman, Martial Hebert
Part of this work analyzes a general formulation for evaluating likelihood of geometric objects.
no code implementations • ICCV 2019 • Pavel Tokmakov, Yu-Xiong Wang, Martial Hebert
One of the key limitations of modern deep learning approaches lies in the amount of data required to train them.
no code implementations • CVPR 2019 • Yubo Zhang, Pavel Tokmakov, Martial Hebert, Cordelia Schmid
A dominant paradigm for learning-based approaches in computer vision is training generic models, such as ResNet for image recognition, or I3D for video understanding, on large datasets and allowing them to discover the optimal representation for the problem at hand.
1 code implementation • 27 Nov 2018 • Wentao Yuan, David Held, Christoph Mertz, Martial Hebert
Recently, neural networks operating on point clouds have shown superior performance on 3D understanding tasks such as shape classification and part segmentation.
no code implementations • 8 Nov 2018 • Kashyap Chitta, Jianwei Feng, Martial Hebert
With our design, the network progressively learns features specific to the target domain using annotation from only the source domain.
5 code implementations • 2 Aug 2018 • Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, Martial Hebert
Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications.
Ranked #1 on
Point Cloud Completion
on Completion3D
no code implementations • CVPR 2018 • Tiancheng Zhi, Bernardo R. Pires, Martial Hebert, Srinivasa G. Narasimhan
Often, multiple cameras are used for cross-spectral imaging, thus requiring image alignment, or disparity estimation in a stereo setting.
1 code implementation • CVPR 2018 • Yu-Xiong Wang, Ross Girshick, Martial Hebert, Bharath Hariharan
Humans can quickly learn new visual concepts, perhaps because they can easily visualize or imagine what novel objects look like from different views.
no code implementations • ICLR 2018 • Hanzhang Hu, Debadeepta Dey, Martial Hebert, J. Andrew Bagnell
We present an approach for anytime predictions in deep neural networks (DNNs).
no code implementations • CVPR 2018 • Ishan Misra, Ross Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, Laurens van der Maaten
We also show that our model asks questions that generalize to state-of-the-art VQA models and to novel test time distributions.
no code implementations • NeurIPS 2017 • Yu-Xiong Wang, Deva Ramanan, Martial Hebert
We cast this problem as transfer learning, where knowledge from the data-rich classes in the head of the distribution is transferred to the data-poor classes in the tail.
1 code implementation • ICLR 2018 • Hanzhang Hu, Debadeepta Dey, Allison Del Giorno, Martial Hebert, J. Andrew Bagnell
Skip connections are increasingly utilized by deep neural networks to improve accuracy and cost-efficiency.
no code implementations • NeurIPS 2017 • Arun Venkatraman, Nicholas Rhinehart, Wen Sun, Lerrel Pinto, Martial Hebert, Byron Boots, Kris M. Kitani, J. Andrew Bagnell
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
no code implementations • 13 Sep 2017 • Allison Del Giorno, J. Andrew Bagnell, Martial Hebert
We demonstrate that this method is able to remove uninformative parts of the feature space for the anomaly detection setting.
no code implementations • 22 Aug 2017 • Hanzhang Hu, Debadeepta Dey, Martial Hebert, J. Andrew Bagnell
Experimentally, the adaptive weights induce more competitive anytime predictions on multiple recognition data-sets and models than non-adaptive approaches including weighing all losses equally.
6 code implementations • ICCV 2017 • Debidatta Dwibedi, Ishan Misra, Martial Hebert
In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort.
no code implementations • CVPR 2017 • Ishan Misra, Abhinav Gupta, Martial Hebert
In this paper, we present a simple method that respects contextuality in order to compose classifiers of known visual concepts.
1 code implementation • ICCV 2017 • Jacob Walker, Kenneth Marino, Abhinav Gupta, Martial Hebert
First we explicitly model the high level structure of active objects in the scene---humans---and use a VAE to model the possible future movements of humans in the pose space.
Ranked #2 on
Human Pose Forecasting
on Human3.6M
(CMD metric)
no code implementations • 1 Mar 2017 • Hanzhang Hu, Wen Sun, Arun Venkatraman, Martial Hebert, J. Andrew Bagnell
To generalize from batch to online, we first introduce the definition of online weak learning edge with which for strongly convex and smooth loss functions, we present an algorithm, Streaming Gradient Boosting (SGB) with exponential shrinkage guarantees in the number of weak learners.
no code implementations • 8 Dec 2016 • Xiaofang Wang, Kris M. Kitani, Martial Hebert
Given a query image, a second positive image and a third negative image, dissimilar to the first two images, we define a contextualized similarity search criteria.
no code implementations • CVPR 2017 • Matthew Trager, Bernd Sturmfels, John Canny, Martial Hebert, Jean Ponce
The rational camera model recently introduced in [19] provides a general methodology for studying abstract nonlinear imaging systems and their multi-view geometry.
no code implementations • NeurIPS 2016 • Yu-Xiong Wang, Martial Hebert
Inspired by the transferability properties of CNNs, we introduce an additional unsupervised meta-training stage that exposes multiple top layer units to a large amount of unlabeled real-world images.
no code implementations • 28 Sep 2016 • Allison Del Giorno, J. Andrew Bagnell, Martial Hebert
We address an anomaly detection setting in which training sequences are unavailable and anomalies are scored independently of temporal ordering.
no code implementations • 1 Aug 2016 • Shreyansh Daftry, J. Andrew Bagnell, Martial Hebert
The ability to transfer knowledge gained in previous tasks into new contexts is one of the most important mechanisms of human learning.
no code implementations • 28 Jul 2016 • Shreyansh Daftry, Sam Zeng, J. Andrew Bagnell, Martial Hebert
As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical.
no code implementations • 25 Jun 2016 • Jacob Walker, Carl Doersch, Abhinav Gupta, Martial Hebert
We show that our method is able to successfully predict events in a wide variety of scenes and can produce multiple different predictions when the future is ambiguous.
no code implementations • CVPR 2016 • Matthew Trager, Martial Hebert, Jean Ponce
Silhouettes provide rich information on three-dimensional shape, since the intersection of the associated visual cones generates the "visual hull", which encloses and approximates the original shape.
4 code implementations • CVPR 2016 • Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, Martial Hebert
In this paper, we propose a principled approach to learn shared representations in ConvNets using multi-task learning.
Ranked #118 on
Semantic Segmentation
on NYU Depth v2
no code implementations • 28 Mar 2016 • Ishan Misra, C. Lawrence Zitnick, Martial Hebert
With this simple task and no semantic labels, we learn a powerful visual representation using a Convolutional Neural Network (CNN).
Ranked #48 on
Self-Supervised Action Recognition
on HMDB51
no code implementations • 27 Jan 2016 • Damien Teney, Martial Hebert
Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation.
no code implementations • ICCV 2015 • Debadeepta Dey, Varun Ramakrishna, Martial Hebert, J. Andrew Bagnell
We present a simple approach for producing a small number of structured visual outputs which have high recall, for a variety of tasks including monocular pose estimation and semantic scene segmentation.
no code implementations • ICCV 2015 • David F. Fouhey, Wajahat Hussain, Abhinav Gupta, Martial Hebert
Do we really need 3D labels in order to learn how to predict 3D?
no code implementations • ICCV 2015 • Matthew Trager, Martial Hebert, Jean Ponce
Given multiple perspective photographs, point correspondences form the "joint image", effectively a replica of three dimensional space distributed across its two-dimensional projections.
no code implementations • CVPR 2015 • Jiyan Pan, Martial Hebert, Takeo Kanade
In this paper, we propose a novel algorithm that infers the 3D layout of building facades from a single 2D image of an urban scene.
no code implementations • CVPR 2015 • Yu-Xiong Wang, Martial Hebert
In this paper, we explore an approach to generating detectors that is radically different from the conventional way of learning a detector from a large corpus of annotated positive and negative data samples.
no code implementations • CVPR 2015 • Ishan Misra, Abhinav Shrivastava, Martial Hebert
We present a semi-supervised approach that localizes multiple unknown object instances in long videos.
no code implementations • 21 May 2015 • Ishan Misra, Abhinav Shrivastava, Martial Hebert
We present a semi-supervised approach that localizes multiple unknown object instances in long videos.
no code implementations • ICCV 2015 • Jacob Walker, Abhinav Gupta, Martial Hebert
Because our CNN model makes no assumptions about the underlying scene, it can predict future optical flow on a diverse set of scenarios.
no code implementations • 24 Nov 2014 • Debadeepta Dey, Kumar Shaurya Shankar, Sam Zeng, Rupesh Mehta, M. Talha Agcayazi, Christopher Eriksen, Shreyansh Daftry, Martial Hebert, J. Andrew Bagnell
Cameras provide a rich source of information while being passive, cheap and lightweight for small and medium Unmanned Aerial Vehicles (UAVs).
no code implementations • 27 Oct 2014 • Nicholas Rhinehart, Jiaji Zhou, Martial Hebert, J. Andrew Bagnell
We present an efficient algorithm with provable performance for building a high-quality list of detections from any candidate set of region-based proposals.
no code implementations • 19 Sep 2014 • Hanzhang Hu, Alexander Grubb, J. Andrew Bagnell, Martial Hebert
We theoretically guarantee that our algorithms achieve near-optimal linear predictions at each budget when a feature group is chosen.
no code implementations • CVPR 2014 • Jacob Walker, Abhinav Gupta, Martial Hebert
In this paper we present a conceptually simple but surprisingly powerful method for visual prediction which combines the effectiveness of mid-level visual elements with temporal modeling.
no code implementations • CVPR 2014 • Peng Zhang, Jiuling Wang, Ali Farhadi, Martial Hebert, Devi Parikh
We show that a surprisingly straightforward and general approach, that we call ALERT, can predict the likely accuracy (or failure) of a variety of computer vision systems semantic segmentation, vanishing point and camera parameter estimation, and image memorability prediction on individual input images.
no code implementations • CVPR 2014 • Jean Ponce, Martial Hebert
When do the visual rays associated with triplets of point correspondences converge, that is, intersect in a common point?
no code implementations • 2 Dec 2013 • Alexander Grubb, Daniel Munoz, J. Andrew Bagnell, Martial Hebert
Structured prediction plays a central role in machine learning applications from computational biology to computer vision.
no code implementations • NeurIPS 2010 • Abhinav Gupta, Martial Hebert, Takeo Kanade, David M. Blei
There has been a recent push in extraction of 3D spatial layout of scenes.
no code implementations • NeurIPS 2009 • Marius Leordeanu, Martial Hebert, Rahul Sukthankar
When applied to MAP inference, the algorithm is a parallel extension of Iterated Conditional Modes (ICM) with climbing and convergence properties that make it a compelling alternative to the sequential ICM.