Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

We present CREST, an approach for causal reasoning in simulation to learn the\nrelevant state space for a robot manipulation policy. Our approach conducts\ninterventions using internal models, which are simulations with approximate\ndynamics and simplified assumptions. These interventions elicit the structure\nbetween the state and action spaces, enabling construction of neural network\npolicies with only relevant states as input. These policies are pretrained\nusing the internal model with domain randomization over the relevant states.\nThe policy network weights are then transferred to the target domain (e.g., the\nreal world) for fine tuning. We perform extensive policy transfer experiments\nin simulation for two representative manipulation tasks: block stacking and\ncrate opening. Our policies are shown to be more robust to domain shifts, more\nsample efficient to learn, and scale to more complex settings with larger state\nspaces. We also show improved zero-shot sim-to-real transfer of our policies\nfor the block stacking task.\n