Search Results for author:
Found 66 papers, 32 papers with code
Learning General-Purpose Controllers via Locally Communicating Sensorimotor Modules
We observe a wide variety of drastically diverse locomotion styles across morphologies as well as centralized coordination emerging via message passing between decentralized modules purely from the reinforcement learning objective.
A Game Theoretic Perspective on Model-Based Reinforcement Learning
We point out that a large class of MBRL algorithms can be viewed as a game between two players: (1) a policy player, which attempts to maximize rewards under the learned model; (2) a model player, which attempts to fit the real-world data collected by the policy player.
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a number of tasks.
Masked Trajectory Models for Prediction, Representation, and Control
We introduce Masked Trajectory Models (MTM) as a generic abstraction for sequential decision making.
Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learning
Without manually-designed controller nor human demonstrations, we demonstrate that with careful Sim2Real considerations, our policies trained with RL in simulation enable two xArm6 robots to solve the U-shape assembly task with a success rate of above90% in simulation, and 50% on real hardware without any additional real-world fine-tuning.
PaLM-E: An Embodied Multimodal Language Model
Large language models excel at a wide range of complex tasks.
Ranked #1 on
Visual Question Answering (VQA)
on OK-VQA
(using extra training data)
Grounded Decoding: Guiding Text Generation with Grounded Models for Robot Control
Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models.
RT-1: Robotics Transformer for Real-World Control at Scale
By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance.
Melting Pot 2.0
Melting Pot is a research tool developed to facilitate work on multi-agent artificial intelligence, and provides an evaluation protocol that measures generalization to novel social partners in a set of canonical test scenarios.
Multi-Environment Pretraining Enables Transfer to Action Limited Datasets
Using massive datasets to train large-scale models has emerged as a dominant approach for broad generalization in natural language and vision applications.
VeLO: Training Versatile Learned Optimizers by Scaling Up
While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers.
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models
Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing impressive zero-shot and few-shot capabilities across a wide range of tasks.
Implicit Offline Reinforcement Learning via Supervised Learning
Despite the benefits of using implicit models to learn robotic skills via BC, offline RL via Supervised Learning algorithms have been limited to explicit models.
Composing Ensembles of Pre-trained Models via Iterative Consensus
Such closed-loop communication enables models to correct errors caused by other models, significantly boosting performance on downstream tasks, e. g. improving accuracy on grade school math problems by 7. 5%, without requiring any model finetuning.
Ranked #1 on
Video Question Answering
on ActivityNet-QA
(Vocabulary Size metric)
Inner Monologue: Embodied Reasoning through Planning with Language Models
We investigate a variety of sources of feedback, such as success detection, scene description, and human interaction.
Learning Iterative Reasoning through Energy Minimization
Finally, we illustrate that our approach can recursively solve algorithmic problems requiring nested reasoning
Multi-Game Decision Transformers
Specifically, we show that a single transformer-based model - with a single set of weights - trained purely offline can play a suite of up to 46 Atari games simultaneously at close-to-human performance.
Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning
Most successes in robotic manipulation have been restricted to single-arm robots, which limits the range of solvable tasks to pick-and-place, insertion, and objects rearrangement.
Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning
Despite the simplicity of this objective, the compositional nature of building diverse blueprints from a set of blocks leads to an explosion of complexity in structures that agents encounter.
Pre-Trained Language Models for Interactive Decision-Making
Together, these results suggest that language modeling induces representations that are useful for modeling not just language, but also goals and plans; these representations can aid learning and generalization even outside of language processing.
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
However, the plans produced naively by LLMs often cannot map precisely to admissible actions.
Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning
We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size.
Unsupervised Learning of Compositional Energy Concepts
In this work, we propose COMET, which discovers and represents concepts as separate energy functions, enabling us to represent both global concepts as well as objects under a unified framework.
The Neural MMO Platform for Massively Multiagent Research
Neural MMO is a computationally accessible research platform that combines large agent populations, long time horizons, open-ended tasks, and modular game systems.
Pretraining for Language Conditioned Imitation with Transformers
We study reinforcement learning (RL) agents which can utilize language inputs.
How to Adapt Your Large-Scale Vision-and-Language Model
Finally, we provide an empirical analysis and recommend general recipes for efficient transfer learning of vision and language models.
Semi-supervised Offline Reinforcement Learning with Pre-trained Decision Transformers
In this paper, we investigate how we can leverage large reward-free (i. e. task-agnostic) offline datasets of prior interactions to pre-train agents that can then be fine-tuned using a small reward-annotated dataset.
Language Model Pre-training Improves Generalization in Policy Learning
Additional experiments explore the role of language-based encodings in these results; we find that it is possible to train a simple adapter layer that maps from observations and action histories to LM embeddings, and thus that language modeling provides an effective initializer even for tasks with no language as input or output.
Implicit Behavioral Cloning
We find that across a wide range of robot policy learning scenarios, treating supervised policy learning with an implicit model generally performs better, on average, than commonly used explicit models.
Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot
Existing evaluation suites for multi-agent reinforcement learning (MARL) do not assess generalization to novel situations as their primary objective (unlike supervised-learning benchmarks).
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Model-Based Reinforcement Learning via Latent-Space Collocation
The resulting latent collocation method (LatCo) optimizes trajectories of latent states, which improves over previously proposed shooting methods for visual model-based RL on tasks with sparse rewards and long-term goals.
Model-based Reinforcement Learning
reinforcement-learning
+1
Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation
We present Brax, an open source library for rigid body simulation with a focus on performance and parallelism on accelerators, written in JAX.
Decision Transformer: Reinforcement Learning via Sequence Modeling
In particular, we present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling.
Ranked #42 on
Atari Games
on Atari 2600 Pong
(using extra training data)
Pretrained Transformers as Universal Computation Engines
We investigate the capability of a transformer pretrained on natural language to generalize to other modalities with minimal finetuning -- in particular, without finetuning of the self-attention and feedforward layers of the residual blocks.
Reset-Free Lifelong Learning with Skill-Space Planning
We propose Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills.
Improved Contrastive Divergence Training of Energy Based Models
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability.
Gamma-Models: Generative Temporal Difference Learning for Infinite-Horizon Prediction
We introduce the gamma-model, a predictive model of environment dynamics with an infinite, probabilistic horizon.
Compositional Visual Generation with Energy Based Models
A vital aspect of human intelligence is the ability to compose increasingly complex concepts out of simpler ideas, enabling both rapid learning and adaptation of knowledge.
Energy-Based Models for Continual Learning
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems.
Rearrangement: A Challenge for Embodied AI
In the rearrangement task, the goal is to bring a given physical environment into a specified state.
Generative Temporal Difference Learning for Infinite-Horizon Prediction
We introduce the $\gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon.
One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control
We observe that a wide variety of drastically diverse locomotion styles across morphologies as well as centralized coordination emerges via message passing between decentralized modules purely from the reinforcement learning objective.
A Game Theoretic Framework for Model Based Reinforcement Learning
Model-based reinforcement learning (MBRL) has recently gained immense interest due to its potential for sample efficiency and ability to incorporate off-policy data.
Model-based Reinforcement Learning
reinforcement-learning
+1
Compositional Visual Generation and Inference with Energy Based Models
A vital aspect of human intelligence is the ability to compose increasingly complex concepts out of simpler ideas, enabling both rapid learning and adaptation of knowledge.
Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks
We present Neural MMO, a massively multiagent game environment inspired by MMOs and discuss our progress on two more general challenges in multiagent systems engineering for AI research: distributed infrastructure and game IO.
Adaptive Online Planning for Continual Lifelong Learning
We study learning control in an online reset-free lifelong learning scenario, where mistakes can compound catastrophically into the future and the underlying dynamics of the environment may change.
Implicit Generation and Modeling with Energy Based Models
Energy based models (EBMs) are appealing due to their generality and simplicity in likelihood modeling, but have been traditionally difficult to train.
Ranked #3 on
Image Generation
on Stacked MNIST
Emergent Tool Use From Multi-Agent Autocurricula
Through multi-agent competition, the simple objective of hide-and-seek, and standard reinforcement learning algorithms at scale, we find that agents create a self-supervised autocurriculum inducing multiple distinct rounds of emergent strategy, many of which require sophisticated tool use and coordination.
Model Based Planning with Energy Based Models
We provide an online algorithm to train EBMs while interacting with the environment, and show that EBMs allow for significantly better online learning than corresponding feed-forward networks.
Neural MMO: A massively multiplayer game environment for intelligent agents
We demonstrate how this platform can be used to study behavior and learning in large populations of neural agents.
Implicit Generation and Generalization in Energy-Based Models
Energy based models (EBMs) are appealing due to their generality and simplicity in likelihood modeling, but have been traditionally difficult to train.
Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources.
Multi-Agent Reinforcement Learning with Multi-Step Generative Models
We consider model-based reinforcement learning (MBRL) in 2-agent, high-fidelity continuous control problems -- an important domain for robots interacting with other agents in the same workspace.
Concept Learning with Energy-Based Models
Many hallmarks of human intelligence, such as generalizing from limited experience, abstract reasoning and planning, analogical reasoning, creative problem solving, and capacity for language require the ability to consolidate experience into concepts, which act as basic building blocks of understanding and reasoning.
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
We study how local trajectory optimization can cope with approximation errors in the value function, and can stabilize and accelerate value function learning.
Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines
To mitigate this issue, we derive a bias-free action-dependent baseline for variance reduction which fully exploits the structural form of the stochastic policy itself and does not make any additional assumptions about the MDP.
Interpretable and Pedagogical Examples
Teachers intentionally pick the most informative examples to show their students.
Emergent Complexity via Multi-Agent Competition
In this paper, we point out that a competitive multi-agent environment trained with self-play can produce behaviors that are far more complex than the environment itself.
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
Ability to continuously learn and adapt from limited experience in nonstationary environments is an important milestone on the path towards general intelligence.
Learning with Opponent-Learning Awareness
We also show that the LOLA update rule can be efficiently calculated using an extension of the policy gradient estimator, making the method suitable for model-free RL.
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
We explore deep reinforcement learning methods for multi-agent domains.
Ranked #1 on
SMAC+
on Def_Infantry_sequential
Emergence of Grounded Compositional Language in Multi-Agent Populations
By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis.
Prediction and Control with Temporal Segment Models
We introduce a method for learning the dynamics of complex nonlinear systems based on deep generative models over temporal segments of states and actions.
A Paradigm for Situated and Goal-Driven Language Learning
A distinguishing property of human intelligence is the ability to flexibly use language in order to communicate complex ideas with other humans in a variety of contexts.
Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
Nevertheless, often the overall gist of what the policy does in simulation remains valid in the real world.
Interactive Control of Diverse Complex Characters with Neural Networks
We present a method for training recurrent neural networks to act as near-optimal feedback controllers.
