EfficientNets are a family of state-of-the-art image classification models\nbased on efficiently scaled convolutional neural networks. Currently,\nEfficientNets can take on the order of days to train; for example, training an\nEfficientNet-B0 model takes 23 hours on a Cloud TPU v2-8 node. In this paper,\nwe explore techniques to scale up the training of EfficientNets on TPU-v3 Pods\nwith 2048 cores, motivated by speedups that can be achieved when training at\nsuch scales. We discuss optimizations required to scale training to a batch\nsize of 65536 on 1024 TPU-v3 cores, such as selecting large batch optimizers\nand learning rate schedules as well as utilizing distributed evaluation and\nbatch normalization techniques. Additionally, we present timing and performance\nbenchmarks for EfficientNet models trained on the ImageNet dataset in order to\nanalyze the behavior of EfficientNets at scale. With our optimizations, we are\nable to train EfficientNet on ImageNet to an accuracy of 83% in 1 hour and 4\nminutes.\n