Reinforcement Learning for Joint Transmit-Sleep Scheduling in Energy-harvesting Wireless Sensor Networks

Abstract

This paper proposes an interactive multi-agent Reinforcement Learning (RL) framework for joint transmit-sleep scheduling in energy-harvesting wireless sensor networks. The scheduling problem is modeled as a Markov Decision Process (MDP) and solved using temporal difference Reinforcement Learning approach. The online learning abilities of RL make the nodes learn a scheduling policy for transmission and sleep so as to minimize the MAC layer packet loss, while maintaining a stable packet queue, in the presence of limited energy budget and heterogeneous traffic patterns. This is accomplished by the joint coordination of two interactive RL agents launched per node to make scheduling decisions. Each node learns the scheduling policy independently and without explicit information sharing. The decentralized nature of the proposed architecture makes the model computationally efficient, scalable with network size, and suitable for resource constrained Sensor and IoT networks. With simulation experiments, the proposed approach is validated for different traffic and network conditions and compared against an existing hybrid sleep-scheduling mechanism.