1 code implementation • 11 Feb 2025 • Xuefeng Liu, Hung T. C. Le, Siyu Chen, Rick Stevens, Zhuoran Yang, Matthew R. Walter, Yuxin Chen
Online reinforcement learning (RL) enhances policies through direct interactions with the environment, but faces challenges related to sample efficiency.
no code implementations • 26 Nov 2024 • Tewodros Ayalew, Xiao Zhang, Kevin Yuanbo Wu, Tianchong Jiang, Michael Maire, Matthew R. Walter
Utilizing this progress prediction as a dense reward together with an adversarial push-back, we show that PROGRESSOR enables robots to learn complex behaviors without any external supervision.
1 code implementation • 21 Aug 2024 • David Yunis, Kumar Kshitij Patel, Samuel Wheeler, Pedro Savarese, Gal Vardi, Karen Livescu, Michael Maire, Matthew R. Walter
We propose an empirical approach centered on the spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to unify and clarify several phenomena in deep learning.
no code implementations • 25 Apr 2024 • Keziah Naggita, Matthew R. Walter, Avrim Blum
In particular, there is a collection of actions that agents can perform that each have their own cost and each provide the agent with different sets of capabilities.
1 code implementation • 29 Mar 2024 • Peng Ding, Jiading Fang, Peng Li, Kangrui Wang, Xiaochen Zhou, Mo Yu, Jing Li, Matthew R. Walter, Hongyuan Mei
The task is question-answering: for each maze, a large language model reads the walkthrough and answers hundreds of mapping and navigation questions such as "How should you go to Attic from West of House?"
no code implementations • 26 Oct 2023 • Takuma Yoneda, Tianchong Jiang, Gregory Shakhnarovich, Matthew R. Walter
A core capability for robot manipulation is reasoning over where and how to stably place objects in cluttered environments.
no code implementations • 3 Oct 2023 • Xuefeng Liu, Takuma Yoneda, Rick L. Stevens, Matthew R. Walter, Yuxin Chen
Integral to RPI are Robust Active Policy Selection (RAPS) and Robust Policy Gradient (RPG), both of which reason over whether to perform state-wise imitation from the oracles or learn from its own value function when the learner's performance surpasses that of the oracles in a specific state.
1 code implementation • 30 Jun 2023 • Takuma Yoneda, Jiading Fang, Peng Li, Huanyu Zhang, Tianchong Jiang, Shengjie Lin, Ben Picker, David Yunis, Hongyuan Mei, Matthew R. Walter
In this paper, we explore a new dimension in which large language models may benefit robotics planning.
1 code implementation • 17 Jun 2023 • Xuefeng Liu, Takuma Yoneda, Chaoqi Wang, Matthew R. Walter, Yuxin Chen
We introduce MAPS and MAPS-SE, a class of policy improvement algorithms that perform imitation learning from multiple suboptimal oracles.
1 code implementation • 22 May 2023 • Jiading Fang, Shengjie Lin, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Adrien Gaidon, Gregory Shakhnarovich, Matthew R. Walter
A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small neural nets instead of collections of images.
no code implementations • 15 Dec 2021 • Takuma Yoneda, Ge Yang, Matthew R. Walter, Bradly Stadie
A robot's deployment environment often involves perceptual changes that differ from what it has experienced during training.
no code implementations • 6 Dec 2021 • Jiading Fang, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon, Matthew R. Walter
Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams.
no code implementations • 21 May 2021 • Matthew R. Walter, Siddharth Patki, Andrea F. Daniele, Ethan Fahnestock, Felix Duvallet, Sachithra Hemachandra, Jean Oh, Anthony Stentz, Nicholas Roy, Thomas M. Howard
This progress now creates an opportunity for robots to operate not only in isolation, but also with and alongside humans in our complex environments.
1 code implementation • 23 Nov 2020 • Tri Huynh, Simon Kornblith, Matthew R. Walter, Michael Maire, Maryam Khademi
While positive pairs can be generated reliably (e. g., as different views of the same image), it is difficult to accurately establish negative pairs, defined as samples from different images regardless of their semantic content or visual features.
no code implementations • ICML Workshop LaReL 2020 • Takuma Yoneda, Matthew R. Walter, Jason Naradowsky
In this work we perform a controlled study of human language use in a competitive team-based game, and search for useful lessons for structuring communication protocol between autonomous agents.
no code implementations • 9 Sep 2020 • Jacopo Tani, Andrea F. Daniele, Gianmarco Bernasconi, Amaury Camus, Aleksandar Petrov, Anthony Courchesne, Bhairav Mehta, Rohit Suri, Tomasz Zaluska, Matthew R. Walter, Emilio Frazzoli, Liam Paull, Andrea Censi
As robotics matures and increases in complexity, it is more necessary than ever that robot autonomy research be reproducible.
no code implementations • 3 Aug 2020 • Zachary W. Robertson, Matthew R. Walter
In contrast, learning from observation offers a way to utilize unlabeled demonstrations (e. g., video) to perform imitation learning.
1 code implementation • Proceedings of Robotics: Science and Systems (RSS) 2020 • Charles Schaff, Matthew R. Walter
Shared autonomy provides an effective framework for human-robot collaboration that takes advantage of the complementary strengths of humans and robots to achieve common goals.
Robotics
1 code implementation • 15 Feb 2020 • Falcon Z. Dai, Matthew R. Walter
At the working heart of policy iteration algorithms commonly used and studied in the discounted setting of reinforcement learning, the policy evaluation step estimates the value of states with samples from a Markov reward process induced by following a Markov policy in a Markov decision process.
no code implementations • 22 Oct 2019 • Siddharth Patki, Ethan Fahnestock, Thomas M. Howard, Matthew R. Walter
Recent advances in data-driven models for grounded language understanding have enabled robots to interpret increasingly complex instructions.
2 code implementations • 1 Aug 2019 • Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z. Dai, Andrea F. Daniele, Mohammadreza Mostajabi, Steven Basart, Matthew R. Walter, Gregory Shakhnarovich
We introduce DIODE, a dataset that contains thousands of diverse high resolution color images with accurate, dense, long-range depth measurements.
no code implementations • 3 Jul 2019 • Falcon Z. Dai, Matthew R. Walter
We propose a new complexity measure for Markov decision processes (MDPs), the maximum expected hitting cost (MEHC).
1 code implementation • ICML 2020 • Tri Huynh, Michael Maire, Matthew R. Walter
We introduce a novel approach to endowing neural networks with emergent, long-term, large-scale memory.
no code implementations • 21 Mar 2019 • Siddharth Patki, Andrea F. Daniele, Matthew R. Walter, Thomas M. Howard
The speed and accuracy with which robots are able to interpret natural language is fundamental to realizing effective human-robot interaction.
3 code implementations • ICLR 2018 • Charles Schaff, David Yunis, Ayan Chakrabarti, Matthew R. Walter
The physical design of a robot and the policy that controls its motion are inherently coupled, and should be determined according to the task and environment.
no code implementations • 4 Apr 2017 • Dong-Ki Kim, Matthew R. Walter
We propose a vision-based method that localizes a ground vehicle using publicly available satellite imagery as the only prior knowledge of the environment.
1 code implementation • 24 Mar 2017 • Charles Schaff, David Yunis, Ayan Chakrabarti, Matthew R. Walter
The accuracy of such a beacon-based localization system depends both on how beacons are distributed in the environment, and how the robot's location is inferred based on noisy and potentially ambiguous measurements.
no code implementations • 21 Nov 2016 • Hongyuan Mei, Mohit Bansal, Matthew R. Walter
We model coherent conversation continuation via RNN-based dialogue models equipped with a dynamic attention mechanism.
no code implementations • 11 Oct 2016 • Andrea F. Daniele, Mohit Bansal, Matthew R. Walter
We first decide which information to share with the user according to their preferences, using a policy trained from human demonstrations via inverse reinforcement learning.
no code implementations • 17 Nov 2015 • Zhengyang Wu, Mohit Bansal, Matthew R. Walter
In this paper, we present a multimodal learning framework that incorporates both visual and lingual information to estimate the structure and parameters that define kinematic models of articulated objects.
no code implementations • 30 Oct 2015 • Hang Chu, Hongyuan Mei, Mohit Bansal, Matthew R. Walter
We propose a method for accurately localizing ground vehicles with the aid of satellite imagery.
1 code implementation • Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015 • Istvan Chung, Oron Propp, Matthew R. Walter, Thomas M. Howard
Natural language interfaces are powerful tools that enables humans and robots to convey information without the need for extensive training or complex graphical interfaces.
1 code implementation • NAACL 2016 • Hongyuan Mei, Mohit Bansal, Matthew R. Walter
We propose an end-to-end, domain-independent neural encoder-aligner-decoder model for selective generation, i. e., the joint task of content selection and surface realization.
1 code implementation • 12 Jun 2015 • Hongyuan Mei, Mohit Bansal, Matthew R. Walter
We propose a neural sequence-to-sequence model for direction following, a task that is essential to realizing effective autonomous agents.
1 code implementation • 5 Feb 2015 • Sudeep Pillai, Matthew R. Walter, Seth Teller
This paper describes a method by which a robot can acquire an object model by capturing depth imagery of the object as a human moves it through its range of motion.