Reward Conditioned Neural Movement Primitives for Population Based Variational Policy Optimization

The aim of this paper is to study the reward based policy exploration problem\nin a supervised learning approach and enable robots to form complex movement\ntrajectories in challenging reward settings and search spaces. For this, the\nexperience of the robot, which can be bootstrapped from demonstrated\ntrajectories, is used to train a novel Neural Processes-based deep network that\nsamples from its latent space and generates the required trajectories given\ndesired rewards. Our framework can generate progressively improved trajectories\nby sampling them from high reward landscapes, increasing the reward gradually.\nVariational inference is used to create a stochastic latent space to sample\nvarying trajectories in generating population of trajectories given target\nrewards. We benefit from Evolutionary Strategies and propose a novel crossover\noperation, which is applied in the self-organized latent space of the\nindividual policies, allowing blending of the individuals that might address\ndifferent factors in the reward function. Using a number of tasks that require\nsequential reaching to multiple points or passing through gaps between objects,\nwe showed that our method provides stable learning progress and significant\nsample efficiency compared to a number of state-of-the-art robotic\nreinforcement learning methods. Finally, we show the real-world suitability of\nour method through real robot execution involving obstacle avoidance.\n