Friendly Training: Neural Networks Can Adapt Data To Make Learning Easier

In the last decade, motivated by the success of Deep Learning, the scientific\ncommunity proposed several approaches to make the learning procedure of Neural\nNetworks more effective. When focussing on the way in which the training data\nare provided to the learning machine, we can distinguish between the classic\nrandom selection of stochastic gradient-based optimization and more involved\ntechniques that devise curricula to organize data, and progressively increase\nthe complexity of the training set. In this paper, we propose a novel training\nprocedure named Friendly Training that, differently from the aforementioned\napproaches, involves altering the training examples in order to help the model\nto better fulfil its learning criterion. The model is allowed to simplify those\nexamples that are too hard to be classified at a certain stage of the training\nprocedure. The data transformation is controlled by a developmental plan that\nprogressively reduces its impact during training, until it completely vanishes.\nIn a sense, this is the opposite of what is commonly done in order to increase\nrobustness against adversarial examples, i.e., Adversarial Training.\nExperiments on multiple datasets are provided, showing that Friendly Training\nyields improvements with respect to informed data sub-selection routines and\nrandom selection, especially in deep convolutional architectures. Results\nsuggest that adapting the input data is a feasible way to stabilize learning\nand improve the generalization skills of the network.\n