A Heuristic Reinforcement Learning Based on State Backtracking Method

Abstract

Since learning action selection strategy is time-consuming due to the reinforcement learning algorithm, a heuristic reinforcement learning algorithm is presented based on the state backtracking reinforcement learning to improve the action selection strategy of the reinforcement learning. The selection strategies of repeated the action are analyzed and compared by state backtracking. A cost function is defined to denote the importance of repetitive actions. A novel heuristic function is given by combing the action-reward with the cost of an action. This algorithm reinforces the important of an action by heuristic function to speed learning and reduces unnecessary explorations by the cost function, so as to steadily improve the learning efficiency. The simulation results of two robot games proves that the algorithm can effectively enhancement the learning rate of Q-learning based on the state backtracking heuristic reinforcement learning method.