Robot control policies learned in simulation do not often transfer well to\nthe real world. Many existing solutions to this sim-to-real problem, such as\nthe Grounded Action Transformation (GAT) algorithm, seek to correct for or\nground these differences by matching the simulator to the real world. However,\nthe efficacy of these approaches is limited if they do not explicitly account\nfor stochasticity in the target environment. In this work, we analyze the\nproblems associated with grounding a deterministic simulator in a stochastic\nreal world environment, and we present examples where GAT fails to transfer a\ngood policy due to stochastic transitions in the target domain. In response, we\nintroduce the Stochastic Grounded Action Transformation(SGAT) algorithm,which\nmodels this stochasticity when grounding the simulator. We find experimentally\nfor both simulated and physical target domains that SGAT can find policies that\nare robust to stochasticity in the target domain\n