Reinforcement learning enabled cooperative spectrum sensing in cognitive radio networks

Abstract

In cognitive radio (CR) networks, fast and accurate spectrum sensing plays a fundamental role in achieving high spectral efficiency. In this paper, a reinforcement learning (RL) enabled cooperative spectrum sensing scheme is proposed for the secondary users (SUs) to determine the scanning order of channels and select the partner for cooperative spectrum sensing. By applying Q-learning approach, each SU learns the occupancy pattern of the primary channels thus forming a dynamic scanning preference list, so as to reduce the scanning overhead and access delay. To improve the detection efficiency in dynamic environment, a discounted upper confidence bound (D-UCB) based cooperation partner selection algorithm is devised wherein each SU learns the time varying detection probability of its neighbors, and selects the one with the potentially highest detection probability as the cooperation partner. Simulation results demonstrate that the proposed cooperative spectrum sensing scheme achieves significant performance gain over various reference algorithms in terms of scanning overhead, access delay, and detection efficiency.