Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning

We present a method for combining multi-agent communication and traditional\ndata-driven approaches to natural language learning, with an end goal of\nteaching agents to communicate with humans in natural language. Our starting\npoint is a language model that has been trained on generic, not task-specific\nlanguage data. We then place this model in a multi-agent self-play environment\nthat generates task-specific rewards used to adapt or modulate the model,\nturning it into a task-conditional language model. We introduce a new way for\ncombining the two types of learning based on the idea of reranking language\nmodel samples, and show that this method outperforms others in communicating\nwith humans in a visual referential communication task. Finally, we present a\ntaxonomy of different types of language drift that can occur alongside a set of\nmeasures to detect them.\n