Consistent Dropout for Policy Gradient Reinforcement Learning
Dropout has long been a staple of supervised learning, but is rarely used in reinforcement learning. We analyze why naive application of dropout is problematic for policy-gradient learning algorithms and introduce consistent dropout, a simple technique to address this instability. We demonstrate consistent dropout enables stable training with A2C and PPO in both continuous and discrete action environments across a wide range of dropout probabilities. Finally, we show that consistent dropout enables the online training of complex architectures such as GPT without needing to disable the model's native dropout.
PDF Abstract
Datasets
Add Datasets
introduced or used in this paper
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
A2C •
Adam •
Attention Dropout •
BPE •
Cosine Annealing •
Dense Connections •
Discriminative Fine-Tuning •
Dropout •
Entropy Regularization •
GELU •
GPT •
Layer Normalization •
Linear Layer •
Linear Warmup With Cosine Annealing •
Multi-Head Attention •
PPO •
Residual Connection •
Scaled Dot-Product Attention •
Softmax •
Weight Decay
