Search Results for author: Aleksandra Faust

Found 48 papers, 13 papers with code

Many-Shot In-Context Learning

no code implementations17 Apr 2024 Rishabh Agarwal, Avi Singh, Lei M. Zhang, Bernd Bohnet, Stephanie Chan, Ankesh Anand, Zaheer Abbas, Azade Nova, John D. Co-Reyes, Eric Chu, Feryal Behbahani, Aleksandra Faust, Hugo Larochelle

Finally, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases and can learn high-dimensional functions with numerical inputs.

Few-Shot Learning In-Context Learning

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL

no code implementations6 Mar 2024 Jesse Farebrother, Jordi Orbay, Quan Vuong, Adrien Ali Taïga, Yevgen Chebotar, Ted Xiao, Alex Irpan, Sergey Levine, Pablo Samuel Castro, Aleksandra Faust, Aviral Kumar, Rishabh Agarwal

Observing this discrepancy, in this paper, we investigate whether the scalability of deep RL can also be improved simply by using classification in place of regression for training value functions.

Atari Games regression +1

Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web

1 code implementation30 Nov 2023 Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, Izzeddin Gur

We show that while existing prompted LMAs (gpt-3. 5-turbo or gpt-4) achieve 94. 0% average success rate on base tasks, their performance degrades to 24. 9% success rate on compositional tasks.

Decision Making Language Modelling

Levels of AGI: Operationalizing Progress on the Path to AGI

no code implementations4 Nov 2023 Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, Shane Legg

With these principles in mind, we propose 'Levels of AGI' based on depth (performance) and breadth (generality) of capabilities, and reflect on how current systems fit into this ontology.

Autonomous Driving

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

no code implementations24 Jul 2023 Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra Faust

Pre-trained large language models (LLMs) have recently achieved better generalization and sample efficiency in autonomous web automation.

 Ranked #1 on on Mind2Web

Code Generation Denoising +3

Personality Traits in Large Language Models

1 code implementation1 Jul 2023 Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, Maja Matarić

The advent of large language models (LLMs) has revolutionized natural language processing, enabling the generation of coherent and contextually relevant human-like text.


Multimodal Web Navigation with Instruction-Finetuned Foundation Models

no code implementations19 May 2023 Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur

The progress of autonomous web navigation has been hindered by the dependence on billions of exploratory interactions via online reinforcement learning, and domain-specific model designs that make it difficult to leverage generalization from rich out-of-domain data.

Autonomous Web Navigation Instruction Following +1

Multi-Agent Reachability Calibration with Conformal Prediction

no code implementations2 Apr 2023 Anish Muthali, Haotian Shen, Sampada Deglurkar, Michael H. Lim, Rebecca Roelofs, Aleksandra Faust, Claire Tomlin

We investigate methods to provide safety assurances for autonomous agents that incorporate predictions of other, uncontrolled agents' behavior into their own trajectory planning.

Autonomous Driving Conformal Prediction +2

Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios

no code implementations21 Dec 2022 Yiren Lu, Justin Fu, George Tucker, Xinlei Pan, Eli Bronstein, Rebecca Roelofs, Benjamin Sapp, Brandyn White, Aleksandra Faust, Shimon Whiteson, Dragomir Anguelov, Sergey Levine

To our knowledge, this is the first application of a combined imitation and reinforcement learning approach in autonomous driving that utilizes large amounts of real-world human driving data.

Autonomous Driving Imitation Learning +2

CLUTR: Curriculum Learning via Unsupervised Task Representation Learning

1 code implementation19 Oct 2022 Abdus Salam Azad, Izzeddin Gur, Jasper Emhoff, Nathaniel Alexis, Aleksandra Faust, Pieter Abbeel, Ion Stoica

Recently, Unsupervised Environment Design (UED) emerged as a new paradigm for zero-shot generalization by simultaneously learning a task distribution and agent policies on the generated tasks.

Reinforcement Learning (RL) Representation Learning +1

Understanding HTML with Large Language Models

no code implementations8 Oct 2022 Izzeddin Gur, Ofir Nachum, Yingjie Miao, Mustafa Safdari, Austin Huang, Aakanksha Chowdhery, Sharan Narang, Noah Fiedel, Aleksandra Faust

We contribute HTML understanding models (fine-tuned LLMs) and an in-depth analysis of their capabilities under three tasks: (i) Semantic Classification of HTML elements, (ii) Description Generation for HTML inputs, and (iii) Autonomous Web Navigation of HTML pages.

Autonomous Web Navigation Decoder +1

Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization

no code implementations25 May 2022 Sungryull Sohn, Hyunjae Woo, Jongwook Choi, lyubing qiang, Izzeddin Gur, Aleksandra Faust, Honglak Lee

Different from the previous meta-rl methods trying to directly infer the unstructured task embedding, our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks, and use it as a prior to improve the task inference in testing.

Hierarchical Reinforcement Learning Meta Reinforcement Learning +2

Environment Generation for Zero-Shot Compositional Reinforcement Learning

1 code implementation NeurIPS 2021 Izzeddin Gur, Natasha Jaques, Yingjie Miao, Jongwook Choi, Manoj Tiwari, Honglak Lee, Aleksandra Faust

We learn to generate environments composed of multiple pages or rooms, and train RL agents capable of completing wide-range of complex tasks in those environments.

Navigate reinforcement-learning +1

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

no code implementations11 Jan 2022 Jack Parker-Holder, Raghu Rajan, Xingyou Song, André Biedenkapp, Yingjie Miao, Theresa Eimer, Baohe Zhang, Vu Nguyen, Roberto Calandra, Aleksandra Faust, Frank Hutter, Marius Lindauer

The combination of Reinforcement Learning (RL) with deep learning has led to a series of impressive feats, with many believing (deep) RL provides a path towards generally capable agents.

AutoML Meta-Learning +2

Compositional Learning-based Planning for Vision POMDPs

1 code implementation17 Dec 2021 Sampada Deglurkar, Michael H. Lim, Johnathan Tucker, Zachary N. Sunberg, Aleksandra Faust, Claire J. Tomlin

The Partially Observable Markov Decision Process (POMDP) is a powerful framework for capturing decision-making problems that involve state and transition uncertainty.

Decision Making

Targeted Environment Design from Offline Data

no code implementations29 Sep 2021 Izzeddin Gur, Ofir Nachum, Aleksandra Faust

We formalize our approach as offline targeted environment design(OTED), which automatically learns a distribution over simulator parameters to match a provided offline dataset, and then uses the learned simulator to train an RL agent in standard online fashion.

Offline RL Reinforcement Learning (RL)

SparseDice: Imitation Learning for Temporally Sparse Data via Regularization

no code implementations ICML Workshop URL 2021 Alberto Camacho, Izzeddin Gur, Marcin Lukasz Moczulski, Ofir Nachum, Aleksandra Faust

We are concerned with a setting where the demonstrations comprise only a subset of state-action pairs (as opposed to the whole trajectories).

Imitation Learning

Differentiable Architecture Search for Reinforcement Learning

1 code implementation4 Jun 2021 Yingjie Miao, Xingyou Song, John D. Co-Reyes, Daiyi Peng, Summer Yue, Eugene Brevdo, Aleksandra Faust

In this paper, we investigate the fundamental question: To what extent are gradient-based neural architecture search (NAS) techniques applicable to RL?

Neural Architecture Search reinforcement-learning +1

Joint Attention for Multi-Agent Coordination and Social Learning

no code implementations15 Apr 2021 Dennis Lee, Natasha Jaques, Chase Kew, Jiaxing Wu, Douglas Eck, Dale Schuurmans, Aleksandra Faust

We then train agents to minimize the difference between the attention weights that they apply to the environment at each timestep, and the attention of other agents.

Inductive Bias Reinforcement Learning (RL)

Adversarial Environment Generation for Learning to Navigate the Web

1 code implementation2 Mar 2021 Izzeddin Gur, Natasha Jaques, Kevin Malta, Manoj Tiwari, Honglak Lee, Aleksandra Faust

The regret objective trains the adversary to design a curriculum of environments that are "just-the-right-challenge" for the navigator agents; our results show that over time, the adversary learns to generate increasingly complex web navigation tasks.

Benchmarking Decision Making +2

AutoPilot: Automating SoC Design Space Exploration for SWaP Constrained Autonomous UAVs

no code implementations5 Feb 2021 Srivatsan Krishnan, Zishen Wan, Kshitij Bhardwaj, Paul Whatmough, Aleksandra Faust, Sabrina Neuman, Gu-Yeon Wei, David Brooks, Vijay Janapa Reddi

Balancing a computing system for a UAV requires considering both the cyber (e. g., sensor rate, compute performance) and physical (e. g., payload weight) characteristics that affect overall performance.

Bayesian Optimization BIG-bench Machine Learning +1

Evolving Reinforcement Learning Algorithms

5 code implementations ICLR 2021 John D. Co-Reyes, Yingjie Miao, Daiyi Peng, Esteban Real, Sergey Levine, Quoc V. Le, Honglak Lee, Aleksandra Faust

Learning from scratch on simple classical control and gridworld tasks, our method rediscovers the temporal-difference (TD) algorithm.

Atari Games Meta-Learning +2

Visual Navigation Among Humans with Optimal Control as a Supervisor

1 code implementation20 Mar 2020 Varun Tolani, Somil Bansal, Aleksandra Faust, Claire Tomlin

Videos describing our approach and experiments, as well as a demo of HumANav are available on the project website.

Navigate Social Navigation +1

Neural Collision Clearance Estimator for Batched Motion Planning

no code implementations14 Oct 2019 J. Chase Kew, Brian Ichter, Maryam Bandari, Tsang-Wei Edward Lee, Aleksandra Faust

We present a neural network collision checking heuristic, ClearanceNet, and a planning algorithm, CN-RRT.

Motion Planning

Learned Critical Probabilistic Roadmaps for Robotic Motion Planning

no code implementations8 Oct 2019 Brian Ichter, Edward Schmerling, Tsang-Wei Edward Lee, Aleksandra Faust

Critical PRMs are demonstrated to achieve up to three orders of magnitude improvement over uniform sampling, while preserving the guarantees and complexity of sampling-based motion planning.

Motion Planning

Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation

no code implementations27 Sep 2019 Xinlei Pan, Tingnan Zhang, Brian Ichter, Aleksandra Faust, Jie Tan, Sehoon Ha

Here, we propose a zero-shot imitation learning approach for training a visual navigation policy on legged robots from human (third-person perspective) demonstrations, enabling high-quality navigation and cost-effective data collection.

Disentanglement Imitation Learning +1

Safe Policy Learning for Continuous Control

no code implementations25 Sep 2019 Yinlam Chow, Ofir Nachum, Aleksandra Faust, Edgar Duenez-Guzman, Mohammad Ghavamzadeh

We study continuous action reinforcement learning problems in which it is crucial that the agent interacts with the environment only through safe policies, i. e.,~policies that keep the agent in desirable situations, both during training and at convergence.

Continuous Control

Learning to Seek: Autonomous Source Seeking with Deep Reinforcement Learning Onboard a Nano Drone Microcontroller

1 code implementation25 Sep 2019 Bardienus P. Duisterhof, Srivatsan Krishnan, Jonathan J. Cruz, Colby R. Banbury, William Fu, Aleksandra Faust, Guido C. H. E. de Croon, Vijay Janapa Reddi

We present fully autonomous source seeking onboard a highly constrained nano quadcopter, by contributing application-specific system and observation feature design to enable inference of a deep-RL policy onboard a nano quadcopter.

Autonomous Navigation Efficient Exploration +1

RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators from RL Policies

no code implementations10 Jul 2019 Hao-Tien Lewis Chiang, Jasmine Hsu, Marek Fiser, Lydia Tapia, Aleksandra Faust

Through the combination of sampling-based planning, a Rapidly Exploring Randomized Tree (RRT) and an efficient kinodynamic motion planner through machine learning, we propose an efficient solution to long-range planning for kinodynamic motion planning.

Motion Planning

Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation

1 code implementation2 Jun 2019 Srivatsan Krishnan, Behzad Boroujerdian, William Fu, Aleksandra Faust, Vijay Janapa Reddi

We find that the trajectories on an embedded Ras-Pi are vastly different from those predicted on a high-end desktop system, resulting in up to 40% longer trajectories in one of the environments.

Benchmarking Reinforcement Learning (RL) +1

Evolving Rewards to Automate Reinforcement Learning

no code implementations18 May 2019 Aleksandra Faust, Anthony Francis, Dar Mehta

Many continuous control tasks have easily formulated objectives, yet using them directly as a reward in reinforcement learning (RL) leads to suboptimal policies.

Continuous Control Hyperparameter Optimization +2

Long-Range Indoor Navigation with PRM-RL

no code implementations25 Feb 2019 Anthony Francis, Aleksandra Faust, Hao-Tien Lewis Chiang, Jasmine Hsu, J. Chase Kew, Marek Fiser, Tsang-Wei Edward Lee

Long-range indoor navigation requires guiding robots with noisy sensors and controls through cluttered environments along paths that span a variety of buildings.

Navigate reinforcement-learning +2

Lyapunov-based Safe Policy Optimization for Continuous Control

1 code implementation28 Jan 2019 Yin-Lam Chow, Ofir Nachum, Aleksandra Faust, Edgar Duenez-Guzman, Mohammad Ghavamzadeh

We formulate these problems as constrained Markov decision processes (CMDPs) and present safe policy optimization algorithms that are based on a Lyapunov approach to solve them.

Continuous Control Robot Navigation

Learning to Navigate the Web

no code implementations ICLR 2019 Izzeddin Gur, Ulrich Rueckert, Aleksandra Faust, Dilek Hakkani-Tur

Even though recent approaches improve the success rate on relatively simple environments with the help of human demonstrations to guide the exploration, they still fail in environments where the set of possible instructions can reach millions.

Instruction Following Meta-Learning +3

Learning Navigation Behaviors End-to-End with AutoRL

no code implementations26 Sep 2018 Hao-Tien Lewis Chiang, Aleksandra Faust, Marek Fiser, Anthony Francis

The policies are trained in small, static environments with AutoRL, an evolutionary automation layer around Reinforcement Learning (RL) that searches for a deep RL reward and neural network architecture with large-scale hyper-parameter optimization.

Motion Planning reinforcement-learning +1

Resilient Computing with Reinforcement Learning on a Dynamical System: Case Study in Sorting

no code implementations25 Sep 2018 Aleksandra Faust, James B. Aimone, Conrad D. James, Lydia Tapia

Robots and autonomous agents often complete goal-based tasks with limited resources, relying on imperfect models and sensor measurements.

Decision Making reinforcement-learning +1

Deep Neural Networks for Swept Volume Prediction Between Configurations

no code implementations29 May 2018 Hao-Tien Lewis Chiang, Aleksandra Faust, Lydia Tapia

Swept Volume (SV), the volume displaced by an object when it is moving along a trajectory, is considered a useful metric for motion planning.

Motion Planning

PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

no code implementations11 Oct 2017 Aleksandra Faust, Oscar Ramirez, Marek Fiser, Kenneth Oslund, Anthony Francis, James Davidson, Lydia Tapia

The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology.

Reinforcement Learning (RL)

Cannot find the paper you are looking for? You can Submit a new open access paper.