Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networks

We introduce Larq Compute Engine, the world's fastest Binarized Neural\nNetwork (BNN) inference engine, and use this framework to investigate several\nimportant questions about the efficiency of BNNs and to design a new\nstate-of-the-art BNN architecture. LCE provides highly optimized\nimplementations of binary operations and accelerates binary convolutions by 8.5\n- 18.5x compared to their full-precision counterparts on Pixel 1 phones. LCE's\nintegration with Larq and a sophisticated MLIR-based converter allow users to\nmove smoothly from training to deployment. By extending TensorFlow and\nTensorFlow Lite, LCE supports models which combine binary and full-precision\nlayers, and can be easily integrated into existing applications. Using LCE, we\nanalyze the performance of existing BNN computer vision architectures and\ndevelop QuickNet, a simple, easy-to-reproduce BNN that outperforms existing\nbinary networks in terms of latency and accuracy on ImageNet. Furthermore, we\ninvestigate the impact of full-precision shortcuts and the relationship between\nnumber of MACs and model latency. We are convinced that empirical performance\nshould drive BNN architecture design and hope this work will facilitate others\nto design, benchmark and deploy binary models.\n