Search Results for author:
Found 36 papers, 26 papers with code
Enhancing Agent Learning through World Dynamics Modeling
Large language models (LLMs) have been increasingly applied to tasks in language understanding and interactive decision-making, with their impressive performance largely attributed to the extensive domain knowledge embedded within them.
IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents
Seamless interaction between AI agents and humans using natural language remains a key goal in AI research.
DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents
However, developing and evaluating an AI agent's capacity for end-to-end scientific reasoning is challenging as running real-world experiments is often prohibitively expensive or infeasible.
Can Language Models Serve as Text-Based World Simulators?
Can current language models themselves serve as world simulators, correctly predicting how actions change different world states, thus bypassing the need for extensive manual coding?
OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Following
Embodied Instruction Following (EIF) is a crucial task in embodied learning, requiring agents to interact with their environment through egocentric observations to fulfill natural language instructions.
Language-guided Skill Learning with Temporal Variational Inference
We present an algorithm for skill discovery from expert demonstrations.
Policy Improvement using Language Feedback Models
We introduce Language Feedback Models (LFMs) that identify desirable behaviour - actions that help achieve tasks specified in the instruction - for imitation learning in instruction following.
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Thus, they can be seen as stochastic language layers in a language network, where the learnable parameters are the natural language prompts at each layer.
ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games
In this work, we investigate the capacity of language models to generate explicit, interpretable, and interactive world models of scientific and common-sense reasoning tasks.
Augmenting Autotelic Agents with Large Language Models
Without relying on any hand-coded goal representations, reward functions or curriculum, we show that LMA3 agents learn to master a large diversity of skills in a task-agnostic text-based environment.
A Song of Ice and Fire: Analyzing Textual Autotelic Agents in ScienceWorld
We show the importance of selectivity from the social peer's feedback; that experience replay needs to over-sample examples of rare goals; and that following self-generated goal sequences where the agent's competence is intermediate leads to significant improvements in final performance.
Collecting Interactive Multi-modal Datasets for Grounded Language Understanding
Human intelligence can remarkably adapt quickly to new tasks and environments.
Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions
The adoption of pre-trained language models to generate action plans for embodied agents is a promising research strategy.
Behavior Cloned Transformers are Neurosymbolic Reasoners
In this work, we explore techniques for augmenting interactive agents with information from symbolic modules, much like humans use tools like calculators and GPS systems to assist with arithmetic and navigation.
TextWorldExpress: Simulating Text Games at One Million Steps Per Second
Text-based games offer a challenging test bed to evaluate virtual agents at language understanding, multi-step problem-solving, and common-sense reasoning.
Automatic Exploration of Textual Environments with Language-Conditioned Autotelic Agents
In this extended abstract we discuss the opportunities and challenges of studying intrinsically-motivated agents for exploration in textual environments.
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions.
Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language
In order to study how agents can be taught to query external knowledge via language, we first introduce two new environments: the grid-world-based Q-BabyAI and the text-based Q-TextWorld.
Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021
The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment.
ScienceWorld: Is your Agent Smarter than a 5th Grader?
We present ScienceWorld, a benchmark to test agents' scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curriculum.
One-Shot Learning from a Demonstration with Hierarchical Latent Language
Humans have the capability, aided by the expressive compositionality of their language, to learn quickly by demonstration.
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
ALFWorld enables the creation of a new BUTLER agent whose abstract knowledge, learned in TextWorld, corresponds directly to concrete, visually grounded actions.
Graph Policy Network for Transferable Active Learning on Graphs
Graph neural networks (GNNs) have been attracting increasing popularity due to their simplicity and effectiveness in a variety of fields.
Learning Dynamic Belief Graphs to Generalize on Text-Based Games
Playing text-based games requires skills in processing natural language and sequential decision making.
Building Dynamic Knowledge Graphs from Text-based Games
We are interested in learning how to update Knowledge Graphs (KG) from text.
Interactive Fiction Games: A Colossal Adventure
A hallmark of human intelligence is the ability to understand and communicate with language.
Unsupervised State Representation Learning in Atari
State representation learning, or the ability to capture latent generative factors of an environment, is crucial for building intelligent agents that can perform a wide variety of tasks.
Towards Solving Text-based Games by Producing Adaptive Action Spaces
To solve a text-based game, an agent needs to formulate valid text commands for a given context and find the ones that lead to success.
Lessons learned in multilingual grounded language learning
Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language.
Revisiting the Hierarchical Multiscale LSTM
Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art language model that learns interpretable structure from character-level input.
TextWorld: A Learning Environment for Text-based Games
We introduce TextWorld, a sandbox learning environment for the training and evaluation of RL agents on text-based games.
Counting to Explore and Generalize in Text-based Games
We propose a recurrent RL agent with an episodic exploration mechanism that helps discovering good policies in text-based game environments.
Z-Forcing: Training Stochastic Recurrent Networks
Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Neural Autoregressive Distribution Estimation
We present Neural Autoregressive Distribution Estimation (NADE) models, which are neural network architectures applied to the problem of unsupervised distribution and density estimation.
An Infinite Restricted Boltzmann Machine
We present a mathematical construction for the restricted Boltzmann machine (RBM) that doesn't require specifying the number of hidden units.
