As we aim to control complex systems, use of a simulator in model-based\nreinforcement learning is becoming more common. However, it has been\nchallenging to overcome the Reality Gap, which comes from nonlinear model bias\nand susceptibility to disturbance. To address these problems, we propose a\nnovel algorithm that combines data-driven system identification approach\n(Gaussian Process) with a Differential-Dynamic-Programming-based robust optimal\ncontrol method (Iterative Linear Quadratic Control). Our algorithm uses the\nsimulator's model as the mean function for a Gaussian Process and learns only\nthe difference between the simulator's prediction and actual observations,\nmaking it a natural hybrid of simulation and real-world observation. We show\nthat our approach quickly corrects incorrect models, comes up with robust\noptimal controllers, and transfers its acquired model knowledge to new tasks\nefficiently.\n