Search Results for author: Hengshuai Yao

Found 28 papers, 5 papers with code

Learning to Accelerate by the Methods of Step-size Planning

no code implementations1 Apr 2022 Hengshuai Yao

In the second part of this paper, we propose a new class of methods of accelerating gradient descent that have some distinctiveness from existing techniques.

Towards Safe, Explainable, and Regulated Autonomous Driving

no code implementations20 Nov 2021 Shahin Atakishiyev, Mohammad Salameh, Hengshuai Yao, Randy Goebel

There has been growing interest in the development and deployment of autonomous vehicles on roads over the last few years, encouraged by the empirical successes of powerful artificial intelligence techniques (AI), especially in the applications of deep learning and reinforcement learning.

Autonomous Driving reinforcement-learning

Exploring the Robustness of Distributional Reinforcement Learning against Noisy State Observations

no code implementations29 Sep 2021 Ke Sun, Yi Liu, Yingnan Zhao, Hengshuai Yao, Shangling Jui, Linglong Kong

In real scenarios, state observations that an agent observes may contain measurement errors or adversarial noises, misleading the agent to take suboptimal actions or even collapse while training.

Distributional Reinforcement Learning reinforcement-learning

Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observations

no code implementations17 Sep 2021 Ke Sun, Yi Liu, Yingnan Zhao, Hengshuai Yao, Shangling Jui, Linglong Kong

In real scenarios, state observations that an agent observes may contain measurement errors or adversarial noises, misleading the agent to take suboptimal actions or even collapse while training.

Density Estimation Distributional Reinforcement Learning +1

Breaking the Deadly Triad with a Target Network

1 code implementation21 Jan 2021 Shangtong Zhang, Hengshuai Yao, Shimon Whiteson

The deadly triad refers to the instability of a reinforcement learning algorithm when it employs off-policy learning, function approximation, and bootstrapping simultaneously.

Q-Learning

Measuring and mitigating interference in reinforcement learning

no code implementations1 Jan 2021 Vincent Liu, Adam M White, Hengshuai Yao, Martha White

Catastrophic interference is common in many network-based learning systems, and many proposals exist for mitigating it.

reinforcement-learning Representation Learning

Beyond Prioritized Replay: Sampling States in Model-Based RL via Simulated Priorities

1 code implementation28 Sep 2020 Jincheng Mei, Yangchen Pan, Martha White, Amir-Massoud Farahmand, Hengshuai Yao

The prioritized Experience Replay (ER) method has attracted great attention; however, there is little theoretical understanding of such prioritization strategy and why they help.

Variance-Reduced Off-Policy Memory-Efficient Policy Search

no code implementations14 Sep 2020 Daoming Lyu, Qi Qi, Mohammad Ghavamzadeh, Hengshuai Yao, Tianbao Yang, Bo Liu

To achieve variance-reduced off-policy-stable policy optimization, we propose an algorithm family that is memory-efficient, stochastically variance-reduced, and capable of learning from off-policy samples.

reinforcement-learning Stochastic Optimization

Beyond Prioritized Replay: Sampling States in Model-Based Reinforcement Learning via Simulated Priorities

no code implementations19 Jul 2020 Jincheng Mei, Yangchen Pan, Amir-Massoud Farahmand, Hengshuai Yao, Martha White

The prioritized Experience Replay (ER) method has attracted great attention; however, there is little theoretical understanding about why it can help and its limitations.

Autonomous Driving Continuous Control +2

Towards a practical measure of interference for reinforcement learning

no code implementations7 Jul 2020 Vincent Liu, Adam White, Hengshuai Yao, Martha White

In this work, we provide a definition of interference for control in reinforcement learning.

reinforcement-learning

Weakly Supervised Few-shot Object Segmentation using Co-Attention with Visual and Semantic Embeddings

no code implementations26 Jan 2020 Mennatullah Siam, Naren Doraiswamy, Boris N. Oreshkin, Hengshuai Yao, Martin Jagersand

Our results show that few-shot segmentation benefits from utilizing word embeddings, and that we are able to perform few-shot segmentation using stacked joint visual semantic processing with weak image-level labels.

Few-Shot Learning One-shot visual object segmentation +3

One-Shot Weakly Supervised Video Object Segmentation

no code implementations18 Dec 2019 Mennatullah Siam, Naren Doraiswamy, Boris N. Oreshkin, Hengshuai Yao, Martin Jagersand

Conventional few-shot object segmentation methods learn object segmentation from a few labelled support images with strongly labelled segmentation masks.

Semantic Segmentation Video Object Segmentation +2

Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation

1 code implementation ICML 2020 Shangtong Zhang, Bo Liu, Hengshuai Yao, Shimon Whiteson

With the help of the emphasis critic and the canonical value function critic, we show convergence for COF-PAC, where the critics are linear and the actor can be nonlinear.

Mapless Navigation among Dynamics with Social-safety-awareness: a reinforcement learning approach from 2D laser scans

no code implementations8 Nov 2019 Jun Jin, Nhat M. Nguyen, Nazmus Sakib, Daniel Graves, Hengshuai Yao, Martin Jagersand

We observe that our method demonstrates time-efficient path planning behavior with high success rate in mapless navigation tasks.

Robotics

Discounted Reinforcement Learning Is Not an Optimization Problem

no code implementations4 Oct 2019 Abhishek Naik, Roshan Shariff, Niko Yasui, Hengshuai Yao, Richard S. Sutton

Discounted reinforcement learning is fundamentally incompatible with function approximation for control in continuing tasks.

reinforcement-learning

Is Fast Adaptation All You Need?

no code implementations3 Oct 2019 Khurram Javed, Hengshuai Yao, Martha White

Gradient-based meta-learning has proven to be highly effective at learning model initializations, representations, and update rules that allow fast adaptation from a few samples.

Incremental Learning Meta-Learning

Three-Head Neural Network Architecture for AlphaZero Learning

no code implementations25 Sep 2019 Chao GAO, Martin Mueller, Ryan Hayward, Hengshuai Yao, Shangling Jui

A three-head network architecture has been recently proposed that can learn a third action-value head on a fixed dataset the same as for two-head net.

Hill Climbing on Value Estimates for Search-control in Dyna

no code implementations18 Jun 2019 Yangchen Pan, Hengshuai Yao, Amir-Massoud Farahmand, Martha White

In this work, we propose to generate such states by using the trajectory obtained from Hill Climbing (HC) the current estimate of the value function.

Model-based Reinforcement Learning

Distributional Reinforcement Learning for Efficient Exploration

no code implementations13 May 2019 Borislav Mavrin, Shangtong Zhang, Hengshuai Yao, Linglong Kong, Kaiwen Wu, Yao-Liang Yu

In distributional reinforcement learning (RL), the estimated distribution of value function models both the parametric and intrinsic uncertainties.

Atari Games Distributional Reinforcement Learning +2

Single-step Options for Adversary Driving

no code implementations20 Mar 2019 Nazmus Sakib, Hengshuai Yao, Hong Zhang, Shangling Jui

In this paper, we use reinforcement learning for safety driving in adversary settings.

reinforcement-learning

Deep Reinforcement Learning with Decorrelation

no code implementations18 Mar 2019 Borislav Mavrin, Hengshuai Yao, Linglong Kong

Further experiments on the losing games show that our decorelation algorithms can win over DQN and QR-DQN with a fined tuned regularization factor.

Atari Games reinforcement-learning +1

ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search

1 code implementation6 Nov 2018 Shangtong Zhang, Hao Chen, Hengshuai Yao

In this paper, we propose an actor ensemble algorithm, named ACE, for continuous control with a deterministic policy in reinforcement learning.

Continuous Control reinforcement-learning +1

QUOTA: The Quantile Option Architecture for Reinforcement Learning

4 code implementations5 Nov 2018 Shangtong Zhang, Borislav Mavrin, Linglong Kong, Bo Liu, Hengshuai Yao

In this paper, we propose the Quantile Option Architecture (QUOTA) for exploration based on recent advances in distributional reinforcement learning (RL).

Decision Making Distributional Reinforcement Learning +1

Practical Issues of Action-conditioned Next Image Prediction

no code implementations8 Feb 2018 Donglai Zhu, Hao Chen, Hengshuai Yao, Masoud Nosrati, Peyman Yadmellat, Yunfei Zhang

Our major finding is that action tiling encoding is the most important factor leading to the remarkable performance of the CDNA model.

Frame SSIM

Universal Option Models

no code implementations NeurIPS 2014 Hengshuai Yao, Csaba Szepesvari, Richard S. Sutton, Joseph Modayil, Shalabh Bhatnagar

We prove that the UOM of an option can construct a traditional option model given a reward function, and the option-conditional return is computed directly by a single dot-product of the UOM with the reward function.

Cannot find the paper you are looking for? You can Submit a new open access paper.