Smooth Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Lujuan Dang
  • Wanli Wang
  • Chi K. Tse
  • Francis C. M. Lau
  • Shiyuan Wang

Related Research Unit(s)

Detail(s)

Original languageEnglish
Journal / PublicationIEEE Transactions on Wireless Communications
Publication statusOnline published - 29 Jun 2022

Abstract

Spectrum sharing in a cognitive radio system involves a secondary user updating transmit power for sharing spectrum with a primary user. The deep Q-network in the framework of deep reinforcement learning achieves transmit power control by a deep neural network for learning a nonlinear mapping from states to Q-values. Since a deep neural network is confronted with noise susceptibility, the deep Q-network produces deteriorative network parameters and volatile Q-values in the presence of contaminated states. In view of the positive effect of kernel least mean square (KLMS) for signal smoothing, we combine KLMS with the deep Q-network for smoothing network-generated outputs. Since an inappropriate step size of KLMS causes under-smoothing or over-smoothing issues, a weighting procedure using past Q-values is proposed for cooperating with KLMS. We assess the incremental ratio of the success rate of the smooth deep Q-network to that of the deep Q-network (RSR) in cognitive radios. Simulations show that RSR has averaged almost over 30% at the early stage of power control. In particular, the maximum RSR reaches almost over 80% or 180% at different scenarios of power control for the primary user. In addition, the smooth deep Q-network achieves an improved success rate in comparison with other algorithms.

Research Area(s)

  • cognitive radio system, deep reinforcement learning, Interference, Kernel, kernel least mean square, Neural networks, Power control, Quality of service, Receivers, Signal to noise ratio, Spectrum sharing, weighting procedure