Mining YouTube - A dataset for learning fine-grained action concepts from webly supervised video data

Action recognition is so far mainly focusing on the problem of classification\nof hand selected preclipped actions and reaching impressive results in this\nfield. But with the performance even ceiling on current datasets, it also\nappears that the next steps in the field will have to go beyond this fully\nsupervised classification. One way to overcome those problems is to move\ntowards less restricted scenarios. In this context we present a large-scale\nreal-world dataset designed to evaluate learning techniques for human action\nrecognition beyond hand-crafted datasets. To this end we put the process of\ncollecting data on its feet again and start with the annotation of a test set\nof 250 cooking videos. The training data is then gathered by searching for the\nrespective annotated classes within the subtitles of freely available videos.\nThe uniqueness of the dataset is attributed to the fact that the whole process\nof collecting the data and training does not involve any human intervention. To\naddress the problem of semantic inconsistencies that arise with this kind of\ntraining data, we further propose a semantical hierarchical structure for the\nmined classes.\n