Smoothing Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios

The spectrum sharing in a cognitive radio system is related with a secondary user sharing common spectrum with a primary user for power transmit without induc- ing harmful inference. The deep reinforcement learning has been considered as an intelligent power control method via an agent continuously interacting with en- vironment. Traditional deep Q-network in the frame work of deep reinforcement learning utilizes a deep neural network for learning a nonlinear function which maps the state or observation to accumulated rewards conditional on current state and agent action also called Q-value. The state or observation in the radio sys- tem is collected from wireless network and corrupted by noises. The deep neural network may therefore yield undesirable result due to the presence of noises and induced degraded network parameters. Considering that the kernel-based adaptive filter is beneficial for adaptive filtering, we aim to apply the kernel-based adaptive filter into traditional deep Q-network for smoothing network outputs. In addi- tion, a weighting approach on the basis of past Q-values also works together with the deep neural network for further network output smoothing. The weighting approach is especially beneficial for alleviating the over-smoothing issue of the kernel-based adaptive filter. Simulation results have shown the efficiency of the proposed smoothing deep Q-network in spectrum sharing in cognitive radios in comparison with traditional deep Q-network.