FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations

Binary neural networks (BNNs) have 1-bit weights and activations. Such\nnetworks are well suited for FPGAs, as their dominant computations are bitwise\narithmetic and the memory requirement is also significantly reduced. However,\ncompared to start-of-the-art compact convolutional neural network (CNN) models,\nBNNs tend to produce a much lower accuracy on realistic datasets such as\nImageNet. In addition, the input layer of BNNs has gradually become a major\ncompute bottleneck, because it is conventionally excluded from binarization to\navoid a large accuracy loss. This work proposes FracBNN, which exploits\nfractional activations to substantially improve the accuracy of BNNs.\nSpecifically, our approach employs a dual-precision activation scheme to\ncompute features with up to two bits, using an additional sparse binary\nconvolution. We further binarize the input layer using a novel thermometer\nencoding. Overall, FracBNN preserves the key benefits of conventional BNNs,\nwhere all convolutional layers are computed in pure binary MAC operations\n(BMACs). We design an efficient FPGA-based accelerator for our novel BNN model\nthat supports the fractional activations. To evaluate the performance of\nFracBNN under a resource-constrained scenario, we implement the entire\noptimized network architecture on an embedded FPGA (Xilinx Ultra96v2). Our\nexperiments on ImageNet show that FracBNN achieves an accuracy comparable to\nMobileNetV2, surpassing the best-known BNN design on FPGAs with an increase of\n28.9% in top-1 accuracy and a 2.5x reduction in model size. FracBNN also\noutperforms a recently introduced BNN model with an increase of 2.4% in top-1\naccuracy while using the same model size. On the embedded FPGA device, FracBNN\ndemonstrates the ability of real-time image classification.\n