S++: A Fast and Deployable Secure-Computation Framework for Privacy-Preserving Neural Network Training

We introduce S++, a simple, robust, and deployable framework for training a\nneural network (NN) using private data from multiple sources, using\nsecret-shared secure function evaluation. In short, consider a virtual third\nparty to whom every data-holder sends their inputs, and which computes the\nneural network: in our case, this virtual third party is actually a set of\nservers which individually learn nothing, even with a malicious (but\nnon-colluding) adversary.\n Previous work in this area has been limited to just one specific activation\nfunction: ReLU, rendering the approach impractical for many use-cases. For the\nfirst time, we provide fast and verifiable protocols for all common activation\nfunctions and optimize them for running in a secret-shared manner. The ability\nto quickly, verifiably, and robustly compute exponentiation, softmax, sigmoid,\netc., allows us to use previously written NNs without modification, vastly\nreducing developer effort and complexity of code. In recent times, ReLU has\nbeen found to converge much faster and be more computationally efficient as\ncompared to non-linear functions like sigmoid or tanh. However, we argue that\nit would be remiss not to extend the mechanism to non-linear functions such as\nthe logistic sigmoid, tanh, and softmax that are fundamental due to their\nability to express outputs as probabilities and their universal approximation\nproperty. Their contribution in RNNs and a few recent advancements also makes\nthem more relevant.\n