Humans read and write hundreds of billions of messages every day. Further,\ndue to the availability of large datasets, large computing systems, and better\nneural network models, natural language processing (NLP) technology has made\nsignificant strides in understanding, proofreading, and organizing these\nmessages. Thus, there is a significant opportunity to deploy NLP in myriad\napplications to help web users, social networks, and businesses. In particular,\nwe consider smartphones and other mobile devices as crucial platforms for\ndeploying NLP models at scale. However, today's highly-accurate NLP neural\nnetwork models such as BERT and RoBERTa are extremely computationally\nexpensive, with BERT-base taking 1.7 seconds to classify a text snippet on a\nPixel 3 smartphone. In this work, we observe that methods such as grouped\nconvolutions have yielded significant speedups for computer vision networks,\nbut many of these techniques have not been adopted by NLP neural network\ndesigners. We demonstrate how to replace several operations in self-attention\nlayers with grouped convolutions, and we use this technique in a novel network\narchitecture called SqueezeBERT, which runs 4.3x faster than BERT-base on the\nPixel 3 while achieving competitive accuracy on the GLUE test set. The\nSqueezeBERT code will be released.\n