Few-Shot and Zero-Shot Semantic Segmentation for Food Images

Abstract

With the popularity of health management applications, awareness of dietary management is increasing. When calculating the number of calories in a dish, discriminating between food regions is an important factor. However, when using deep learning, a large amount of data is required for training, and it is impractical to collect data for countless food categories. In recent years, a method called few-shot segmentation has been studied to learn a semantic segmentation model using a small amount of training data. In this study, we propose a few-shot and zero-shot segmentation model which targets food images to overcome the insufficient amount of food training data and show the effectiveness of the proposed model on a semantic segmentation task for new food classes. In the proposed model, we employed the word embedding pretrained with a large-scale recipe text dataset, which results in better accuracy than the previous methods.