Joint Robust Beamforming Design for WPT-assisted D2D Communications in MISO-NOMA: Fractional Programming and Deep Reinforcement Learning

This paper proposes a scheme for the envisioned sixth-generation (6G) ultra-massive Machine Type Communications(umMTC). In particular, wireless power transfer (WPT) assisted communication is deployed in non-orthogonal multiple access (NOMA) downlink networks to realize spectrum and energy cooperation. This paper focuses on joint robust beamforming design to maximize the energy efficiency of WPT-assisted D2D communications in multiple-input single-output (MISO)-NOMA downlink networks. To efficiently address the formulated non-concave energy efficiency maximization problem, a pure fractional programming (PFP) algorithm is proposed, where the time switching coefficient of the WPT device and the beamforming vectors of the base station are alternatively optimized by applying the Dinkelbach method and quadratic transform respectively. To prove the optimality of the proposed algorithm, the partial exhaustive search algorithm is proposed as a benchmark. A deep reinforcement learning (DRL)-based method is also applied to directly solve the non-concave problem. The proposed PFP algorithm and the DDPG-based algorithm are compared in the presence of different channel estimation errors. Simulation results show that the proposed PFP algorithm outperforms the DDPG-based algorithm if perfect channel state information (CSI) can be obtained or just have minor errors, while the DDPG-based algorithm is more robust when the channel estimation accuracy is unsatisfactory. On the other hand, one can conclude that the NOMA scheme can provide a higher gain than OMA on the energy efficiency of the WPT-assisted D2D communication in legacy multi-user downlink networks.