Deep Reinforcement Learning for Online Offloading in Wireless Powered Mobile-Edge Computing Networks

In this paper, we consider a wireless powered mobile-edge computing (MEC) network following the binary offloading policy, such that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm under time-varying wireless channels that jointly optimizes task offloading decisions and resource allocation to maximize the data processing capability of the network. In practice, this requires successively solving hard combinatorial optimization problems to address the multi-user correlation in the offloading decisions. The existing approaches are mainly based on either branch-and-bound algorithm or relaxation heuristics, which are limited by the tradeoff between optimality and efficiency. To tackle this problem, we propose in this paper a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network to generate offloading decisions. In particular, the proposed DROO framework does not require any manually labeled training data as the input, and thus completely removes the need of solving combinatorial optimization problems. Besides, it avoids the curse of dimensionality problem encountered by some existing reinforcement learning approaches and is computationally efficient in large-size networks. To further reduce the computational complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing methods. For example, the complexity is reduced from several seconds to less than $0.1$ second in a $30$-user network, making real-time and optimal offloading design truly viable even in a fast fading environment.