Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration

In modern industrial settings with small batch sizes it should be easy to set\nup a robot system for a new task. Strategies exist, e.g. the use of skills, but\nwhen it comes to handling forces and torques, these systems often fall short.\nWe introduce an approach that provides a combination of task-level planning\nwith targeted learning of scenario-specific parameters for skill-based systems.\nWe propose the following pipeline: (1) the user provides a task goal in the\nplanning language PDDL, (2) a plan (i.e., a sequence of skills) is generated\nand the learnable parameters of the skills are automatically identified. An\noperator then chooses (3) reward functions and hyperparameters for the learning\nprocess. Two aspects of our methodology are critical: (a) learning is tightly\nintegrated with a knowledge framework to support symbolic planning and to\nprovide priors for learning, (b) using multi-objective optimization. This can\nhelp to balance key performance indicators (KPIs) such as safety and task\nperformance since they can often affect each other. We adopt a multi-objective\nBayesian optimization approach and learn entirely in simulation. We demonstrate\nthe efficacy and versatility of our approach by learning skill parameters for\ntwo different contact-rich tasks. We show their successful execution on a real\n7-DOF KUKA-iiwa manipulator and outperform the manual parameterization by human\nrobot operators.\n