PropNEAT -- Efficient GPU-Compatible Backpropagation over NeuroEvolutionary Augmenting Topology Networks

We introduce PropNEAT, a fast backpropagation implementation of NEAT that uses a bidirectional mapping of the genome graph to a layer-based architecture that preserves the NEAT genomes whilst enabling efficient GPU backpropagation. We test PropNEAT on 58 binary classification datasets from the Penn Machine Learning Benchmarks database, comparing the performance against logistic regression, dense neural networks and random forests, as well as a densely retrained variant of the final PropNEAT model. Random forests was the highest performer, with no significant difference between it, PropNEAT, PropNEAT-retrain or dense neural networks. Logistic regression was significantly worse than all other models. PropNEAT was faster than a naive NEAT backpropagation method, and both were faster and had better performance than the original NEAT implementation. The per-epoch training time for PropNEAT scales linearly with network depth, and is efficient on GPU implementations for backpropagation. This implementation could be extended to support reinforcement learning or convolutional networks, and is able to find well-performing, sparser and smaller networks with potential for applications in low-power contexts.