Given the current trend of increasing size and complexity of machine learning architectures, it has become of critical importance to identify new approaches to improve the computational efficiency of model training. In this context, we address the advantages of floating-point over fixed-point representation, and present an in-depth study on the use of 8-bit floating-point number formats for activations, weights, and gradients for both training and inference. We explore the effect of different bit-widths for exponents and significands and different exponent biases. The experimental results demonstrate that a suitable choice of these low-precision formats enables faster training and reduced power consumption without any degradation in accuracy for a range of deep learning models for image classification and language processing. computational efficiency and communication efficiency in the exchange of data processing units. This the recent advances in the of low-precision numerical formats to enable reli-able training of modern deep architectures with power efficient computations, and provides a com-prehensive study of the effect of the use of low-precision formats on the training and generalization performance of different model architectures. We also address the trade-off of floating-point formats versus integer formats, and the effect of low-precision arithmetic on non-convex optimization and generalization.