A Martingale Approach and Time-Consistent Sampling-based Algorithms for Risk Management in Stochastic Optimal Control

In this paper, we consider a class of stochastic optimal control problems\nwith risk constraints that are expressed as bounded probabilities of failure\nfor particular initial states. We present here a martingale approach that\ndiffuses a risk constraint into a martingale to construct time-consistent\ncontrol policies. The martingale stands for the level of risk tolerance over\ntime. By augmenting the system dynamics with the controlled martingale, the\noriginal risk-constrained problem is transformed into a stochastic target\nproblem. We extend the incremental Markov Decision Process (iMDP) algorithm to\napproximate arbitrarily well an optimal feedback policy of the original problem\nby sampling in the augmented state space and computing proper boundary\nconditions for the reformulated problem. We show that the algorithm is both\nprobabilistically sound and asymptotically optimal. The performance of the\nproposed algorithm is demonstrated on motion planning and control problems\nsubject to bounded probability of collision in uncertain cluttered\nenvironments.\n