no code implementations • 17 Jun 2024 • Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic, Steven L. Waslander, Yue Wang, Sanja Fidler, Marco Pavone, Peter Karkus
We propose DistillNeRF, a self-supervised learning framework addressing the challenge of understanding 3D environments from limited 2D observations in autonomous driving.
no code implementations • 27 Oct 2023 • Yuxiao Chen, Sushant Veer, Peter Karkus, Marco Pavone
In particular, IJP jointly optimizes over the behavior of the ego and the surrounding agents and leverages deep-learned prediction models as prediction priors that the join trajectory optimization tries to stay close to.
no code implementations • 3 Apr 2023 • Fan-Yun Sun, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
no code implementations • 13 Dec 2022 • Peter Karkus, Boris Ivanovic, Shie Mannor, Marco Pavone
To enable the joint optimization of AV stacks while retaining modularity, we present DiffStack, a differentiable and modular stack for prediction, planning, and control.
no code implementations • 9 Nov 2022 • Tarun Gupta, Peter Karkus, Tong Che, Danfei Xu, Marco Pavone
Effectively exploring the environment is a key challenge in reinforcement learning (RL).
no code implementations • 26 Oct 2022 • Filippos Christianos, Peter Karkus, Boris Ivanovic, Stefano V. Albrecht, Marco Pavone
Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles.
no code implementations • CVPR 2021 • Peter Karkus, Shaojun Cai, David Hsu
We introduce the Differentiable SLAM Network (SLAM-net) along with a navigation architecture to enable planar robot navigation in previously unseen indoor environments.
no code implementations • 3 Oct 2020 • Peter Karkus, Mehdi Mirza, Arthur Guez, Andrew Jaegle, Timothy Lillicrap, Lars Buesing, Nicolas Heess, Theophane Weber
We explore whether integrated tasks like Mujoban can be solved by composing RL modules together in a sense-plan-act hierarchy, where modules have well-defined roles similarly to classic robot architectures.
1 code implementation • 11 Sep 2020 • Mehdi Mirza, Andrew Jaegle, Jonathan J. Hunt, Arthur Guez, Saran Tunyasuvunakool, Alistair Muldal, Théophane Weber, Peter Karkus, Sébastien Racanière, Lars Buesing, Timothy Lillicrap, Nicolas Heess
To encourage progress towards this goal we introduce a set of physically embedded planning problems and make them publicly available.
no code implementations • 19 May 2020 • Peter Karkus, Anelia Angelova, Vincent Vanhoucke, Rico Jonschkowski
We address these tasks by combining spatial structure (differentiable mapping) and end-to-end learning in a novel neural network architecture: the Differentiable Mapping Network (DMN).
1 code implementation • ICLR 2020 • Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee, Nan Ye
The particle filter maintains a belief using learned discriminative update, which is trained end-to-end for decision making.
1 code implementation • 30 May 2019 • Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee
Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data.
no code implementations • 28 May 2019 • Peter Karkus, Xiao Ma, David Hsu, Leslie Pack Kaelbling, Wee Sun Lee, Tomas Lozano-Perez
This paper introduces the Differentiable Algorithm Network (DAN), a composable architecture for robot learning systems.
no code implementations • 26 Apr 2019 • Robert Pinsler, Peter Karkus, Andras Kupcsik, David Hsu, Wee Sun Lee
Our key observation is that experience can be directly generalized over target contexts.
no code implementations • 17 Jul 2018 • Peter Karkus, David Hsu, Wee Sun Lee
We propose to take a novel approach to robot system design where each building block of a larger system is represented as a differentiable program, i. e. a deep neural network.
2 code implementations • 23 May 2018 • Peter Karkus, David Hsu, Wee Sun Lee
Particle filtering is a powerful approach to sequential state estimation and finds application in many domains, including robot localization, object tracking, etc.
2 code implementations • NeurIPS 2017 • Peter Karkus, David Hsu, Wee Sun Lee
It is a recurrent policy network, but it represents a policy for a parameterized set of tasks by connecting a model with a planning algorithm that solves the model, thus embedding the solution structure of planning in a network learning architecture.
no code implementations • 6 Dec 2016 • Peter Karkus, Andras Kupcsik, David Hsu, Wee Sun Lee
Scarce data is a major challenge to scaling robot learning to truly complex tasks, as we need to generalize locally learned policies over different "contexts".