A C Code Generator for Fast Inference and Simple Deployment of Convolutional Neural Networks on Resource Constrained Systems

Inference of Convolutional Neural Networks in time critical applications\nusually requires a GPU. In robotics or embedded devices these are often not\navailable due to energy, space and cost constraints. Furthermore, installation\nof a deep learning framework or even a native compiler on the target platform\nis not possible. This paper presents a neural network code generator (NNCG)\nthat generates from a trained CNN a plain ANSI C code file that encapsulates\nthe inference in single a function. It can easily be included in existing\nprojects and due to lack of dependencies, cross compilation is usually\npossible. Additionally, the code generation is optimized based on the known\ntrained CNN and target platform following four design principles. The system is\nevaluated utilizing small CNN designed for this application. Compared to\nTensorFlow XLA and Glow speed-ups of up to 11.81 can be shown and even GPUs are\noutperformed regarding latency.\n