Impact of Subjectivity in Deep Reinforcement Learning based Defense of Cloud Storage

Abstract

Cloud storage is a target of advanced persistent threats (APTs), where a sophisticated adversary attempts to steal sensitive data in a continuous manner. Human monitoring and intervention is an integral part of the reinforcement learning (RL) approaches to defend against APTs. In this paper, prospect theory (PT) is used to model the subjective behavior of the cloud storage defender in assigning computing resources (processing units) to scan and monitor the cloud storage system against an APT attacker bot, which attempts to steal information from the cloud. Under a constraint on the total number of processing units and a lack of knowledge of the opponent’s resource allocation strategy, we study the defense performance of a federated maximum-likelihood deep Q-network (FMLDQ) RL algorithm against a sophisticated branching dueling deep Q-network (BDQ) RL attack algorithm. Specifically, the RL strategy for the defender is affected by subjective decisions in estimating the processing units of the attacker. Simulation results show that when the defender has more resources than the attacker, an EUT-based defense strategy (without human intervention) yields better data protection. On the other hand, when the defender has fewer resources, a PT-based defense strategy (with human intervention) is better.