Representation learning from videos in-the-wild: An object-centric approach

We propose a method to learn image representations from uncurated videos. We\ncombine a supervised loss from off-the-shelf object detectors and\nself-supervised losses which naturally arise from the video-shot-frame-object\nhierarchy present in each video. We report competitive results on 19 transfer\nlearning tasks of the Visual Task Adaptation Benchmark (VTAB), and on 8\nout-of-distribution-generalization tasks, and discuss the benefits and\nshortcomings of the proposed approach. In particular, it improves over the\nbaseline on all 18/19 few-shot learning tasks and 8/8 out-of-distribution\ngeneralization tasks. Finally, we perform several ablation studies and analyze\nthe impact of the pretrained object detector on the performance across this\nsuite of tasks.\n