Improving Human-Object Interaction Detection via Phrase Learning and Label Composition

Human-Object Interaction (HOI) detection is a fundamental task in high-level\nhuman-centric scene understanding. We propose PhraseHOI, containing a HOI\nbranch and a novel phrase branch, to leverage language prior and improve\nrelation expression. Specifically, the phrase branch is supervised by semantic\nembeddings, whose ground truths are automatically converted from the original\nHOI annotations without extra human efforts. Meanwhile, a novel label\ncomposition method is proposed to deal with the long-tailed problem in HOI,\nwhich composites novel phrase labels by semantic neighbors. Further, to\noptimize the phrase branch, a loss composed of a distilling loss and a balanced\ntriplet loss is proposed. Extensive experiments are conducted to prove the\neffectiveness of the proposed PhraseHOI, which achieves significant improvement\nover the baseline and surpasses previous state-of-the-art methods on Full and\nNonRare on the challenging HICO-DET benchmark.\n