Search Results for author: Michael L. Littman

Found 44 papers, 14 papers with code

Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy

1 code implementation10 Jul 2024 Cameron Allen, Aaron Kirtland, Ruo Yu Tao, Sam Lobel, Daniel Scott, Nicholas Petrocelli, Omer Gottesman, Ronald Parr, Michael L. Littman, George Konidaris

Our metric, the $\lambda$-discrepancy, is the difference between two distinct temporal difference (TD) value estimates, each computed using TD($\lambda$) with a different value of $\lambda$.

Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages

1 code implementation3 Jul 2024 Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, Stephen H. Bach

To bridge this gap, we introduce \benchmarkName, a benchmark designed to evaluate language models' ability to generate PDDL code from natural language descriptions of planning tasks.

Language Modelling valid

Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex

1 code implementation26 Nov 2022 Charles Lovering, Jessica Zosa Forde, George Konidaris, Ellie Pavlick, Michael L. Littman

AlphaZero, an approach to reinforcement learning that couples neural networks and Monte Carlo tree search (MCTS), has produced state-of-the-art strategies for traditional board games like chess, Go, shogi, and Hex.

Board Games

Reward-Predictive Clustering

no code implementations7 Nov 2022 Lucas Lehnert, Michael J. Frank, Michael L. Littman

Recent advances in reinforcement-learning research have demonstrated impressive results in building algorithms that can out-perform humans in complex tasks.

Clustering reinforcement-learning +1

Designing Rewards for Fast Learning

no code implementations30 May 2022 Henry Sowerby, Zhiyuan Zhou, Michael L. Littman

To solve this optimization problem, we propose a linear-programming based algorithm that efficiently finds a reward function that maximizes action gap and minimizes subjective discount.

Q-Learning Reinforcement Learning (RL)

Faster Deep Reinforcement Learning with Slower Online Network

1 code implementation10 Dec 2021 Kavosh Asadi, Rasool Fakoor, Omer Gottesman, Taesup Kim, Michael L. Littman, Alexander J. Smola

In this paper we endow two popular deep reinforcement learning algorithms, namely DQN and Rainbow, with updates that incentivize the online network to remain in the proximity of the target network.

reinforcement-learning Reinforcement Learning (RL)

On the Expressivity of Markov Reward

no code implementations NeurIPS 2021 David Abel, Will Dabney, Anna Harutyunyan, Mark K. Ho, Michael L. Littman, Doina Precup, Satinder Singh

We then provide a set of polynomial-time algorithms that construct a Markov reward function that allows an agent to optimize tasks of each of these three types, and correctly determine when no such reward function exists.

People construct simplified mental representations to plan

no code implementations14 May 2021 Mark K. Ho, David Abel, Carlos G. Correa, Michael L. Littman, Jonathan D. Cohen, Thomas L. Griffiths

We propose a computational account of this simplification process and, in a series of pre-registered behavioral experiments, show that it is subject to online cognitive control and that people optimally balance the complexity of a task representation and its utility for planning and acting.

Towards Sample Efficient Agents through Algorithmic Alignment

1 code implementation7 Aug 2020 Mingxuan Li, Michael L. Littman

We demonstrate the potential of graph neural network in supporting sample efficient learning by showing that Deep Graph Value Network can outperform unstructured baselines by a large margin in solving the Markov Decision Process (MDP).

Graph Neural Network Reinforcement Learning (RL)

Context-Driven Satirical News Generation

no code implementations WS 2020 Zachary Horvitz, Nam Do, Michael L. Littman

While mysterious, humor likely hinges on an interplay of entities, their relationships, and cultural connotations.

News Generation

The Efficiency of Human Cognition Reflects Planned Information Processing

no code implementations13 Feb 2020 Mark K. Ho, David Abel, Jonathan D. Cohen, Michael L. Littman, Thomas L. Griffiths

Thus, people should plan their actions, but they should also be smart about how they deploy resources used for planning their actions.

Learning State Abstractions for Transfer in Continuous Control

2 code implementations8 Feb 2020 Kavosh Asadi, David Abel, Michael L. Littman

In this work, we answer this question in the affirmative, where we take "simple learning algorithm" to be tabular Q-Learning, the "good representations" to be a learned state abstraction, and "challenging problems" to be continuous control tasks.

Continuous Control Q-Learning +2

Lipschitz Lifelong Reinforcement Learning

1 code implementation15 Jan 2020 Erwan Lecarpentier, David Abel, Kavosh Asadi, Yuu Jinnai, Emmanuel Rachelson, Michael L. Littman

We consider the problem of knowledge transfer when an agent is facing a series of Reinforcement Learning (RL) tasks.

reinforcement-learning Reinforcement Learning (RL) +1

Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging

1 code implementation8 Dec 2019 John R. Zech, Jessica Zosa Forde, Michael L. Littman

Averaging predictions from 10 models reduced variability by nearly 70% (mean coefficient of variation from 0. 543 to 0. 169, t-test 15. 96, p-value < 0. 0001).

Interactive Learning of Environment Dynamics for Sequential Tasks

no code implementations19 Jul 2019 Robert Loftin, Bei Peng, Matthew E. Taylor, Michael L. Littman, David L. Roberts

In order for robots and other artificial agents to efficiently learn to perform useful tasks defined by an end user, they must understand not only the goals of those tasks, but also the structure and dynamics of that user's environment.

Deep Reinforcement Learning from Policy-Dependent Human Feedback

no code implementations12 Feb 2019 Dilip Arumugam, Jun Ki Lee, Sophie Saskin, Michael L. Littman

To widen their accessibility and increase their utility, intelligent agents must be able to learn complex behaviors as specified by (non-expert) human users.

reinforcement-learning Reinforcement Learning (RL)

Mitigating Planner Overfitting in Model-Based Reinforcement Learning

no code implementations3 Dec 2018 Dilip Arumugam, David Abel, Kavosh Asadi, Nakul Gopalan, Christopher Grimm, Jun Ki Lee, Lucas Lehnert, Michael L. Littman

An agent with an inaccurate model of its environment faces a difficult choice: it can ignore the errors in its model and act in the real world in whatever way it determines is optimal with respect to its model.

Model-based Reinforcement Learning Position +2

Towards a Simple Approach to Multi-step Model-based Reinforcement Learning

no code implementations31 Oct 2018 Kavosh Asadi, Evan Cater, Dipendra Misra, Michael L. Littman

When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes.

Model-based Reinforcement Learning reinforcement-learning +1

Transfer with Model Features in Reinforcement Learning

no code implementations4 Jul 2018 Lucas Lehnert, Michael L. Littman

Further, we present a Successor Feature model which shows that learning Successor Features is equivalent to learning a Model-Reduction.

reinforcement-learning Reinforcement Learning (RL)

Lipschitz Continuity in Model-based Reinforcement Learning

1 code implementation ICML 2018 Kavosh Asadi, Dipendra Misra, Michael L. Littman

We go on to prove an error bound for the value-function estimate arising from Lipschitz models and show that the estimated value function is itself Lipschitz.

Model-based Reinforcement Learning reinforcement-learning +1

Learning Approximate Stochastic Transition Models

1 code implementation26 Oct 2017 Yuhang Song, Christopher Grimm, Xianming Wang, Michael L. Littman

We examine the problem of learning mappings from state to state, suitable for use in a model-based reinforcement-learning setting, that simultaneously generalize to novel states and can capture stochastic transitions.

Model-based Reinforcement Learning reinforcement-learning +1

Summable Reparameterizations of Wasserstein Critics in the One-Dimensional Setting

no code implementations19 Sep 2017 Christopher Grimm, Yuhang Song, Michael L. Littman

Generative adversarial networks (GANs) are an exciting alternative to algorithms for solving density estimation problems---using data to assess how likely samples are to be drawn from the same distribution.

Density Estimation

Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning

no code implementations31 Jul 2017 Lucas Lehnert, Stefanie Tellex, Michael L. Littman

One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks.

reinforcement-learning Reinforcement Learning (RL)

Environment-Independent Task Specifications via GLTL

no code implementations14 Apr 2017 Michael L. Littman, Ufuk Topcu, Jie Fu, Charles Isbell, Min Wen, James Macglashan

We propose a new task-specification language for Markov decision processes that is designed to be an improvement over reward functions by being environment independent.

reinforcement-learning Reinforcement Learning (RL)

Interactive Learning from Policy-Dependent Human Feedback

no code implementations ICML 2017 James MacGlashan, Mark K. Ho, Robert Loftin, Bei Peng, Guan Wang, David Roberts, Matthew E. Taylor, Michael L. Littman

This paper investigates the problem of interactively learning behaviors communicated by a human teacher using positive and negative feedback.

Near Optimal Behavior via Approximate State Abstraction

1 code implementation15 Jan 2017 David Abel, D. Ellis Hershkowitz, Michael L. Littman

The combinatorial explosion that plagues planning and reinforcement learning (RL) algorithms can be moderated using state abstraction.

reinforcement-learning Reinforcement Learning (RL)

An Alternative Softmax Operator for Reinforcement Learning

1 code implementation ICML 2017 Kavosh Asadi, Michael L. Littman

A softmax operator applied to a set of values acts somewhat like the maximization function and somewhat like an average.

Decision Making reinforcement-learning +1

Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes

no code implementations6 Feb 2013 Anthony R. Cassandra, Michael L. Littman, Nevin Lianwen Zhang

Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into another.

Graphical Models for Game Theory

no code implementations10 Jan 2013 Michael Kearns, Michael L. Littman, Satinder Singh

The interpretation is that the payoff to player i is determined entirely by the actions of player i and his neighbors in the graph, and thus the payoff matrix to player i is indexed only by these players.

Online Linear Regression and Its Application to Model-Based Reinforcement Learning

no code implementations NeurIPS 2007 Alexander L. Strehl, Michael L. Littman

We provide a provably efficient algorithm for learning Markov Decision Processes (MDPs) with continuous state and action spaces in the online setting.

Model-based Reinforcement Learning regression +2

Cannot find the paper you are looking for? You can Submit a new open access paper.