Deep Reinforcement Learning and Game Theory for Computation Offloading in Dynamic Edge Computing Markets

Abstract

As a promising paradigm, computation offloading technology can offload computing tasks to multi-access edge computing (MEC) servers, which is an appealing choice for resource-constrained terminal devices to reduce their computational effort. However, due to limited resources, one crucial research challenge for computation offloading is to design the appropriate offloading policy to determine which tasks should be offloaded in some complex circumstances. In this paper, we study the offloading decision problem in a software-defined networking (SDN) driven MEC environment with multiple users and multiple servers. To ensure that end-users do not abuse the computing resources in the MEC system, we formulate the profit of MEC servers as our optimization objective. We jointly optimize the selection of MEC servers, the size of offloading data, and the price of MEC computing service to maximize the profit of MEC servers. However, considering the dynamic and stochastic of end-users, it is challenging to obtain the optimal policy in such a MEC environment. We apply deep reinforcement learning (DRL) and Game theory to our proposed approach. Specifically, we propose a proximal policy optimization (PPO) reinforcement learning framework to tackle the selection of MEC servers. Secondly, a two-step optimization problem is formulated to determine the size of offloading data and the pricing of computing services. The optimal values of those two were determined by achieving the Nash equilibrium of the strategy game between end-users. Extensive simulation results prove that our proposal has a better performance than existing solutions in convergence time and stability.