Target-dependent UNITER: A Transformer-Based Multimodal Language Comprehension Model for Domestic Service Robots

Currently, domestic service robots have an insufficient ability to interact\nnaturally through language. This is because understanding human instructions is\ncomplicated by various ambiguities and missing information. In existing\nmethods, the referring expressions that specify the relationships between\nobjects are insufficiently modeled. In this paper, we propose Target-dependent\nUNITER, which learns the relationship between the target object and other\nobjects directly by focusing on the relevant regions within an image, rather\nthan the whole image. Our method is an extension of the UNITER-based\nTransformer that can be pretrained on general-purpose datasets. We extend the\nUNITER approach by introducing a new architecture for handling the target\ncandidates. Our model is validated on two standard datasets, and the results\nshow that Target-dependent UNITER outperforms the baseline method in terms of\nclassification accuracy.\n