An efficient bandwidth allocation scheme for hierarchical cellular networks with energy harvesting: an actor-critic approach

Abstract

ABSTRACT In this paper, we propose an efficient bandwidth allocation strategy for multiclass services in hierarchical cellular networks that consist of an operation controller, several small-cell base stations (SBSs), and a number of mobile users. Each SBS is equipped with a finite-capacity battery that is regularly recharged by a solar harvester. We aims to find the optimal bandwidth allocation policy in order to enhance the network performance in terms of user satisfaction and energy efficiency under energy harvesting and bandwidth sharing constraints. Since the arrivals of harvested energy and traffic requests are unknown due to users’ mobility and stochastic request generation, it is necessary to design a learning framework for the controller in order to predict these dynamics through interaction with the environment. For this purpose, we first formulate the resource allocation problem as the framework of a Markov decision process, and then, we employ an actor-critic algorithm to find the optimal policy under which the controller can effectively allocate the limited bandwidth to the SBSs for their data transmissions. We evaluate the performance of the proposed scheme through comprehensive simulations with different settings, and show that the proposed bandwidth allocation scheme can enhance the network’s performance in the long run.