Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks

Abstract

: Reinforcement learning (RL) has shown promising performance in several applications such as robotics and games. However, the use of RL in emerging real-world domains such as smart industry and asset management remains scarce. This paper addresses the problem of optimal maintenance planning using historical data. We propose a novel Deep RL (DRL) framework based on Graph Convolutional Networks (GCN) to leverage the inherent graph structure of typical assets. As demonstrator, we employ an underground sewer pipe network. In particular, instead of dispersed maintenance actions of individual pipes across the network, the GCN ensures the grouping of maintenance actions of geographically close pipes. We perform experiments using the distinct physical characteristics, deterioration profiles, and historical data of sewer inspections within an urban environment. The results show that combining Deep Q-Networks (DQN) with GCN leads to structurally more reliable networks and a higher degree of maintenance grouping, compared to DQN with fully-connected layers and standard preventive and corrective maintenance strategy that are often adopted in practice. Our approach shows potential for developing efficient and practical maintenance plans in terms of cost and reliability.