TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Continuous Control	acrobot.swingup	SMuZero	Return	417.52	# 1
Continuous Control	ball_in_cup.catch	SMuZero	Return	977.38	# 1
Continuous Control	cartpole.balance	SMuZero	Return	984.86	# 1
Continuous Control	cartpole.balance_sparse	SMuZero	Return	998.14	# 1
Continuous Control	cartpole.swingup	SMuZero	Return	868.87	# 1
Continuous Control	cartpole.swingup_sparse	SMuZero	Return	846.91	# 1
Continuous Control	cheetah.run	SMuZero	Return	914.39	# 1
Continuous Control	finger.spin	SMuZero	Return	986.38	# 1
Continuous Control	finger.turn_easy	SMuZero	Return	972.53	# 1
Continuous Control	finger.turn_hard	SMuZero	Return	963.07	# 1
Continuous Control	hopper.hop	SMuZero	Return	528.24	# 1
Continuous Control	hopper.stand	SMuZero	Return	926.5	# 1
Continuous Control	pendulum.swingup	SMuZero	Return	837.76	# 1
Continuous Control	quadruped.run	SMuZero	Return	923.54	# 1
Continuous Control	quadruped.walk	SMuZero	Return	933.77	# 1
Continuous Control	reacher.easy	SMuZero	Return	982.26	# 1
Continuous Control	reacher.hard	SMuZero	Return	971.53	# 1
Continuous Control	walker.run	SMuZero	Return	931.06	# 1
Continuous Control	walker.stand	SMuZero	Return	987.79	# 1
Continuous Control	walker.walk	SMuZero	Return	975.46	# 1

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-acrobot-swingup)](https://paperswithcode.com/sota/continuous-control-on-acrobot-swingup?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-ball-in-cup-catch-2)](https://paperswithcode.com/sota/continuous-control-on-ball-in-cup-catch-2?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-cartpole-balance)](https://paperswithcode.com/sota/continuous-control-on-cartpole-balance?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-cartpole-balance-sparse)](https://paperswithcode.com/sota/continuous-control-on-cartpole-balance-sparse?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-cartpole-swingup-2)](https://paperswithcode.com/sota/continuous-control-on-cartpole-swingup-2?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-cartpole-swingup-sparse)](https://paperswithcode.com/sota/continuous-control-on-cartpole-swingup-sparse?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-cheetah-run-2)](https://paperswithcode.com/sota/continuous-control-on-cheetah-run-2?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-finger-spin-2)](https://paperswithcode.com/sota/continuous-control-on-finger-spin-2?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-finger-turn-easy)](https://paperswithcode.com/sota/continuous-control-on-finger-turn-easy?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-finger-turn-hard)](https://paperswithcode.com/sota/continuous-control-on-finger-turn-hard?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-hopper-hop)](https://paperswithcode.com/sota/continuous-control-on-hopper-hop?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-hopper-stand)](https://paperswithcode.com/sota/continuous-control-on-hopper-stand?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-pendulum-swingup)](https://paperswithcode.com/sota/continuous-control-on-pendulum-swingup?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-quadruped-run)](https://paperswithcode.com/sota/continuous-control-on-quadruped-run?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-quadruped-walk)](https://paperswithcode.com/sota/continuous-control-on-quadruped-walk?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-reacher-easy-2)](https://paperswithcode.com/sota/continuous-control-on-reacher-easy-2?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-reacher-hard)](https://paperswithcode.com/sota/continuous-control-on-reacher-hard?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-walker-run)](https://paperswithcode.com/sota/continuous-control-on-walker-run?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-walker-stand)](https://paperswithcode.com/sota/continuous-control-on-walker-stand?p=learning-and-planning-in-complex-action)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/learning-and-planning-in-complex-action/continuous-control-on-walker-walk-2)](https://paperswithcode.com/sota/continuous-control-on-walker-walk-2?p=learning-and-planning-in-complex-action)`

Learning and Planning in Complex Action Spaces

13 Apr 2021 · Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Mohammadamin Barekatain, Simon Schmitt, David Silver ·

Many important real-world problems have action spaces that are high-dimensional, continuous or both, making full enumeration of all possible actions infeasible. Instead, only small subsets of actions can be sampled for the purpose of policy evaluation and improvement. In this paper, we propose a general framework to reason in a principled way about policy evaluation and improvement over such sampled action subsets. This sample-based policy iteration framework can in principle be applied to any reinforcement learning algorithm based upon policy iteration. Concretely, we propose Sampled MuZero, an extension of the MuZero algorithm that is able to learn in domains with arbitrarily complex action spaces by planning over sampled actions. We demonstrate this approach on the classical board game of Go and on two continuous control benchmark domains: DeepMind Control Suite and Real-World RL Suite.

PDF Abstract

Code

Add Remove Mark official

opendilab/LightZero

↳ Quickstart in

Spaces

869

Tasks

Add Remove

Continuous Control

Game of Go

Datasets

DeepMind Control Suite

Results from the Paper

Edit

Ranked #1 on Continuous Control on acrobot.swingup

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Continuous Control	acrobot.swingup	SMuZero	Return	417.52	# 1	Compare
Continuous Control	ball_in_cup.catch	SMuZero	Return	977.38	# 1	Compare
Continuous Control	cartpole.balance	SMuZero	Return	984.86	# 1	Compare
Continuous Control	cartpole.balance_sparse	SMuZero	Return	998.14	# 1	Compare
Continuous Control	cartpole.swingup	SMuZero	Return	868.87	# 1	Compare
Continuous Control	cartpole.swingup_sparse	SMuZero	Return	846.91	# 1	Compare
Continuous Control	cheetah.run	SMuZero	Return	914.39	# 1	Compare
Continuous Control	finger.spin	SMuZero	Return	986.38	# 1	Compare
Continuous Control	finger.turn_easy	SMuZero	Return	972.53	# 1	Compare
Continuous Control	finger.turn_hard	SMuZero	Return	963.07	# 1	Compare
Continuous Control	hopper.hop	SMuZero	Return	528.24	# 1	Compare
Continuous Control	hopper.stand	SMuZero	Return	926.5	# 1	Compare
Continuous Control	pendulum.swingup	SMuZero	Return	837.76	# 1	Compare
Continuous Control	quadruped.run	SMuZero	Return	923.54	# 1	Compare
Continuous Control	quadruped.walk	SMuZero	Return	933.77	# 1	Compare
Continuous Control	reacher.easy	SMuZero	Return	982.26	# 1	Compare
Continuous Control	reacher.hard	SMuZero	Return	971.53	# 1	Compare
Continuous Control	walker.run	SMuZero	Return	931.06	# 1	Compare
Continuous Control	walker.stand	SMuZero	Return	987.79	# 1	Compare
Continuous Control	walker.walk	SMuZero	Return	975.46	# 1	Compare

Methods

Add Remove

Average Pooling • Batch Normalization • Convolution • Monte-Carlo Tree Search • MuZero • Prioritized Experience Replay • ReLU • Residual Block • Residual Connection

Edit Social Preview

Learning and Planning in Complex Action Spaces

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove