Neuromorphic Multiply-And-Accumulate (MAC) circuits utilizing synaptic weight\nelements based on SRAM or novel Non-Volatile Memories (NVMs) provide a\npromising approach for highly efficient hardware representations of neural\nnetworks. NVM density and robustness requirements suggest that off-line\ntraining is the right choice for "edge" devices, since the requirements for\nsynapse precision are much less stringent. However, off-line training using\nideal mathematical weights and activations can result in significant loss of\ninference accuracy when applied to non-ideal hardware. Non-idealities such as\nmulti-bit quantization of weights and activations, non-linearity of weights,\nfinite max/min ratios of NVM elements, and asymmetry of positive and negative\nweight components all result in degraded inference accuracy. In this work, it\nis demonstrated that non-ideal Multi-Layer Perceptron (MLP) architectures using\nlow bitwidth weights and activations can be trained with negligible loss of\ninference accuracy relative to their Floating Point-trained counterparts using\na proposed off-line, continuously differentiable HW-aware training algorithm.\nThe proposed algorithm is applicable to a wide range of hardware models, and\nuses only standard neural network training methods. The algorithm is\ndemonstrated on the MNIST and EMNIST datasets, using standard MLPs.\n