Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions

Class labels used for machine learning are relatable to each other, with\ncertain class labels being more similar to each other than others (e.g. images\nof cats and dogs are more similar to each other than those of cats and cars).\nSuch similarity among classes is often the cause of poor model performance due\nto the models confusing between them. Current labeling techniques fail to\nexplicitly capture such similarity information. In this paper, we instead\nexploit the similarity between classes by capturing the similarity information\nwith our novel confidence labels. Confidence labels are probabilistic labels\ndenoting the likelihood of similarity, or confusability, between the classes.\nOften even after models are trained to differentiate between classes in the\nfeature space, the similar classes' latent space still remains clustered. We\nview this type of clustering as valuable information and exploit it with our\nnovel projective loss functions. Our projective loss functions are designed to\nwork with confidence labels with an ability to relax the loss penalty for\nerrors that confuse similar classes. We use our approach to train neural\nnetworks with noisy labels, as we believe noisy labels are partly a result of\nconfusability arising from class similarity. We show improved performance\ncompared to the use of standard loss functions. We conduct a detailed analysis\nusing the CIFAR-10 dataset and show our proposed methods' applicability to\nlarger datasets, such as ImageNet and Food-101N.\n