Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise

The growing importance of massive datasets used for deep learning makes\nrobustness to label noise a critical property for classifiers to have. Sources\nof label noise include automatic labeling, non-expert labeling, and label\ncorruption by data poisoning adversaries. Numerous previous works assume that\nno source of labels can be trusted. We relax this assumption and assume that a\nsmall subset of the training data is trusted. This enables substantial label\ncorruption robustness performance gains. In addition, particularly severe label\nnoise can be combated by using a set of trusted data with clean labels. We\nutilize trusted data by proposing a loss correction technique that utilizes\ntrusted examples in a data-efficient manner to mitigate the effects of label\nnoise on deep neural network classifiers. Across vision and natural language\nprocessing tasks, we experiment with various label noises at several strengths,\nand show that our method significantly outperforms existing methods.\n