Representation Matters: Improving Perception and Exploration for Robotics

Projecting high-dimensional environment observations into lower-dimensional\nstructured representations can considerably improve data-efficiency for\nreinforcement learning in domains with limited data such as robotics. Can a\nsingle generally useful representation be found? In order to answer this\nquestion, it is important to understand how the representation will be used by\nthe agent and what properties such a 'good' representation should have. In this\npaper we systematically evaluate a number of common learnt and hand-engineered\nrepresentations in the context of three robotics tasks: lifting, stacking and\npushing of 3D blocks. The representations are evaluated in two use-cases: as\ninput to the agent, or as a source of auxiliary tasks. Furthermore, the value\nof each representation is evaluated in terms of three properties:\ndimensionality, observability and disentanglement. We can significantly improve\nperformance in both use-cases and demonstrate that some representations can\nperform commensurate to simulator states as agent inputs. Finally, our results\nchallenge common intuitions by demonstrating that: 1) dimensionality strongly\nmatters for task generation, but is negligible for inputs, 2) observability of\ntask-relevant aspects mostly affects the input representation use-case, and 3)\ndisentanglement leads to better auxiliary tasks, but has only limited benefits\nfor input representations. This work serves as a step towards a more systematic\nunderstanding of what makes a 'good' representation for control in robotics,\nenabling practitioners to make more informed choices for developing new learned\nor hand-engineered representations.\n