An Unsupervised Domain Adaptation Scheme for Single-Stage Artwork Recognition in Cultural Sites

Recognizing artworks in a cultural site using images acquired from the user's\npoint of view (First Person Vision) allows to build interesting applications\nfor both the visitors and the site managers. However, current object detection\nalgorithms working in fully supervised settings need to be trained with large\nquantities of labeled data, whose collection requires a lot of times and high\ncosts in order to achieve good performance. Using synthetic data generated from\nthe 3D model of the cultural site to train the algorithms can reduce these\ncosts. On the other hand, when these models are tested with real images, a\nsignificant drop in performance is observed due to the differences between real\nand synthetic images. In this study we consider the problem of Unsupervised\nDomain Adaptation for object detection in cultural sites. To address this\nproblem, we created a new dataset containing both synthetic and real images of\n16 different artworks. We hence investigated different domain adaptation\ntechniques based on one-stage and two-stage object detector, image-to-image\ntranslation and feature alignment. Based on the observation that single-stage\ndetectors are more robust to the domain shift in the considered settings, we\nproposed a new method which builds on RetinaNet and feature alignment that we\ncalled DA-RetinaNet. The proposed approach achieves better results than\ncompared methods on the proposed dataset and on Cityscapes. To support research\nin this field we release the dataset at the following link\nhttps://iplab.dmi.unict.it/EGO-CH-OBJ-UDA/ and the code of the proposed\narchitecture at https://github.com/fpv-iplab/DA-RetinaNet.\n