Learning Versatile Skills with Curriculum Masking

1 code implementation • 23 Oct 2024 • Yao Tang, Zhihui Xie, Zichuan Lin, Deheng Ye, Shuai Li

Motivated by how humans learn by organizing knowledge in a curriculum, CurrMask adjusts its masking scheme during pretraining for learning versatile skills.

Decision Making Offline RL +2

Replay-enhanced Continual Reinforcement Learning

1 code implementation • 20 Nov 2023 • Tiantian Zhang, Kevin Zehua Shen, Zichuan Lin, Bo Yuan, Xueqian Wang, Xiu Li, Deheng Ye

On the other hand, offline learning on replayed tasks while learning a new task may induce a distributional shift between the dataset and the learned policy on old tasks, resulting in forgetting.

Continual Learning reinforcement-learning +1

Future-conditioned Unsupervised Pretraining for Decision Transformer

1 code implementation • 26 May 2023 • Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, Shuai Li

While promising, return conditioning is limited to training data labeled with rewards and therefore faces challenges in learning from unsupervised data.

Decision Making Reinforcement Learning (RL)

Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimization

1 code implementation • 5 Feb 2023 • Zichuan Lin, Xiapeng Wu, Mingfei Sun, Deheng Ye, Qiang Fu, Wei Yang, Wei Liu

Recent success in Deep Reinforcement Learning (DRL) methods has shown that policy optimization with respect to an off-policy distribution via importance sampling is effective for sample reuse.

Deep Reinforcement Learning

A Survey on Transformers in Reinforcement Learning

no code implementations • 8 Jan 2023 • Wenzhe Li, Hao Luo, Zichuan Lin, Chongjie Zhang, Zongqing Lu, Deheng Ye

Transformer has been considered the dominating neural architecture in NLP and CV, mostly under supervised settings.

reinforcement-learning Reinforcement Learning +2

Pretraining in Deep Reinforcement Learning: A Survey

no code implementations • 8 Nov 2022 • Zhihui Xie, Zichuan Lin, Junyou Li, Shuai Li, Deheng Ye

The past few years have seen rapid progress in combining reinforcement learning (RL) with deep learning.

Deep Reinforcement Learning reinforcement-learning +2

Revisiting Discrete Soft Actor-Critic

1 code implementation • 21 Sep 2022 • Haibin Zhou, Tong Wei, Zichuan Lin, Junyou Li, Junliang Xing, Yuanchun Shi, Li Shen, Chao Yu, Deheng Ye

We study the adaption of Soft Actor-Critic (SAC), which is considered as a state-of-the-art reinforcement learning (RL) algorithm, from continuous action space to discrete action space.

Atari Games Q-Learning +1

Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization

no code implementations • 1 Sep 2022 • Tiantian Zhang, Zichuan Lin, Yuxing Wang, Deheng Ye, Qiang Fu, Wei Yang, Xueqian Wang, Bin Liang, Bo Yuan, Xiu Li

A key challenge of continual reinforcement learning (CRL) in dynamic environments is to promptly adapt the RL agent's behavior as the environment changes over its lifetime, while minimizing the catastrophic forgetting of the learned information.

Bayesian Inference Knowledge Distillation +4

MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned

no code implementations • 17 Feb 2022 • Anssi Kanervisto, Stephanie Milani, Karolis Ramanauskas, Nicholay Topin, Zichuan Lin, Junyou Li, Jianing Shi, Deheng Ye, Qiang Fu, Wei Yang, Weijun Hong, Zhongyue Huang, Haicheng Chen, Guangjun Zeng, Yue Lin, Vincent Micheli, Eloi Alonso, François Fleuret, Alexander Nikulin, Yury Belousov, Oleg Svidchenko, Aleksei Shpilman

With this in mind, we hosted the third edition of the MineRL ObtainDiamond competition, MineRL Diamond 2021, with a separate track in which we permitted any solution to promote the participation of newcomers.

JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning

no code implementations • 7 Dec 2021 • Zichuan Lin, Junyou Li, Jianing Shi, Deheng Ye, Qiang Fu, Wei Yang

To address this, we propose JueWu-MC, a sample-efficient hierarchical RL approach equipped with representation learning and imitation learning to deal with perception and exploration.

Efficient Exploration Hierarchical Reinforcement Learning +6

Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

no code implementations • 9 Jun 2021 • Zichuan Lin, Jing Huang, BoWen Zhou, Xiaodong He, Tengyu Ma

Recent work (Takanobu et al., 2020) proposed the system-wise evaluation on dialog systems and found that improvement on individual components (e. g., NLU, policy) in prior work may not necessarily bring benefit to pipeline systems in system-wise evaluation.

Data Augmentation Goal-Oriented Dialog

RD$^2$: Reward Decomposition with Representation Decomposition

no code implementations • NeurIPS 2020 • Zichuan Lin, Derek Yang, Li Zhao, Tao Qin, Guangwen Yang, Tie-Yan Liu

In this work, we propose a set of novel reward decomposition principles by constraining uniqueness and compactness of different state features/representations relevant to different sub-rewards.

Model-based Adversarial Meta-Reinforcement Learning

1 code implementation • NeurIPS 2020 • Zichuan Lin, Garrett Thomas, Guangwen Yang, Tengyu Ma

When the test task distribution is different from the training task distribution, the performance may degrade significantly.

continuous-control Continuous Control +4

Distributional Reward Decomposition for Reinforcement Learning

no code implementations • NeurIPS 2019 • Zichuan Lin, Li Zhao, Derek Yang, Tao Qin, Guangwen Yang, Tie-Yan Liu

Many reinforcement learning (RL) tasks have specific properties that can be leveraged to modify existing RL algorithms to adapt to those tasks and further improve performance, and a general class of such properties is the multiple reward channel.

reinforcement-learning Reinforcement Learning +1

Fully Parameterized Quantile Function for Distributional Reinforcement Learning

6 code implementations • NeurIPS 2019 • Derek Yang, Li Zhao, Zichuan Lin, Tao Qin, Jiang Bian, Tie-Yan Liu

The key challenge in practical distributional RL algorithms lies in how to parameterize estimated distributions so as to better approximate the true continuous distribution.

Atari Games Distributional Reinforcement Learning +3

Object-Oriented Dynamics Learning through Multi-Level Abstraction

1 code implementation • 16 Apr 2019 • Guangxiang Zhu, Jianhao Wang, Zhizhou Ren, Zichuan Lin, Chongjie Zhang

We also design a spatial-temporal relational reasoning mechanism for MAOP to support instance-level dynamics learning and handle partial observability.

Object Relational Reasoning +1

Episodic Memory Deep Q-Networks

no code implementations • 19 May 2018 • Zichuan Lin, Tianqi Zhao, Guangwen Yang, Lintao Zhang

Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN).

Atari Games Reinforcement Learning +1