NPC Control Across Algorithm Classes: Static Logic, Evolutionary, Reinforcement Learning, and Hybrid Generative Methods

Abstract

Non-player character (NPC) control in video games spans distinct classes of methods: Static logic, evolutionary search, gradient-based reinforcement learning, and hybrids with generative models. We present a cross-class comparison in a single 2D platformer we created using the Unity game engine of five representative controllers: a designer-authored Behavior Tree (BT; static logic), a Genetic Algorithm (GA; evolutionary parameter search), a Genetic Programming–evolved Behavior Tree (EBT; evolutionary program synthesis), deep Reinforcement Learning (RL) with Proximal Policy Optimization (PPO), and a Variational Autoencoder–initialized RL agent (VAE–RL; a hybrid that uses a VAE to warm-start PPO). To our knowledge, this is the first side-by-side evaluation of BT, GA, EBT, deep RL, and VAE–RL within the same game environment under a shared observation/action interface, matched training budget, and unified metrics. We evaluate each controller on three levels of increasing difficulty, running ten episodes per level. Primary metrics are total game score (from collectibles, enemy defeats and time) and completion rate. Results reveal class-dependent trade-offs: VAE-RL attains the highest scores with perfect reliability across all levels; BT is also perfectly reliable but slightly lower-scoring; deep RL reaches competitive means yet shows variability (about 70% completion on the harder levels); EBT is strong on easier content but fails occasionally on the hardest level; and GA collapses on the trap-heavy medium level before recovering on the hard level. We interpret these outcomes along key design axes: static versus dynamic control, white-box versus black-box policies, search-driven versus gradient-driven learning and hand-crafted versus data-driven design, and offer guidance on selecting an algorithm class for similar games.