Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks

We are approaching a future where social robots will progressively become\nwidespread in many aspects of our daily lives, including education, healthcare,\nwork, and personal use. All of such practical applications require that humans\nand robots collaborate in human environments, where social interaction is\nunavoidable. Along with verbal communication, successful social interaction is\nclosely coupled with the interplay between nonverbal perception and action\nmechanisms, such as observation of gaze behaviour and following their\nattention, coordinating the form and function of hand gestures. Humans perform\nnonverbal communication in an instinctive and adaptive manner, with no effort.\nFor robots to be successful in our social landscape, they should therefore\nengage in social interactions in a humanlike way, with increasing levels of\nautonomy. In particular, nonverbal gestures are expected to endow social robots\nwith the capability of emphasizing their speech, or showing their intentions.\nMotivated by this, our research sheds a light on modeling human behaviors in\nsocial interactions, specifically, forecasting human nonverbal social signals\nduring dyadic interactions, with an overarching goal of developing robotic\ninterfaces that can learn to imitate human dyadic interactions. Such an\napproach will ensure the messages encoded in the robot gestures could be\nperceived by interacting partners in a facile and transparent manner, which\ncould help improve the interacting partner perception and makes the social\ninteraction outcomes enhanced.\n