Hierarchical reinforcement learning from competitive self-play for dual-aircraft formation air combat

Abstract

The recent development of technology helps in the revolutionary war and it controls the war which is influenced by brilliant planning. The maneuver aircraft of intelligent algorithm aid the pilot to decide the particular position on the battlefield. Nowadays the hardware components of radar and missiles are widely used and the Beyond-Visual Range (BVR) is the most popular method applied in air combat. The introduction of close-range air combat maneuver decisions generates the attention of researchers in artificial intelligence. Most of the existing methods are based on autonomous aircraft focused in air combat scenario but manual air combats are widely applied in dual aircraft. Based on the factors mentioned above, a novel hierarchical maneuver decision architecture applied to a dual-aircraft close-range air combat scenario. Subsequently, the Soft Actor-Critic (SAC) algorithm is merged with competitive self-play which integrates the knowledge of sub-strategies. Further, the reinforcement learning technique is employed to achieve an approximate Nash equilibrium master strategy. The experimental results show that the hierarchical architecture exhibits good performance, symmetry, and robustness. The research generates a solution for intelligent formation of air combat in the future and guidance for Manned or unmanned aircraft cooperative combat.