TRAINING NETWORK WITH DISCRETE WEIGHT VALUES

Some embodiments provide a method for configuring a machine-trained (MT) network that includes input nodes, output nodes, and interior nodes between the input and output nodes. Each node produces an output value and each interior node and output node receives as input values a set of output values of other nodes and applies weights to each received input value. The weights are configurable parameters for training. The method propagates a set of inputs through the MT network to generate a set of outputs. Each input has a corresponding expected output. The method calculates a value of a continuously-differentiable augmented loss function that combines a measurement of a difference between each output and its corresponding expected output and a term that biases training of the weights towards a set of discrete values. The method trains the weights by backpropagating a gradient of the continuously-differentiable augmented loss function at the calculated value.