Theoretical and Numerical Analysis of Approximate Dynamic Programming with Approximation Errors

This study is aimed at answering the famous question of how the approximation\nerrors at each iteration of Approximate Dynamic Programming (ADP) affect the\nquality of the final results considering the fact that errors at each iteration\naffect the next iteration. To this goal, convergence of Value Iteration scheme\nof ADP for deterministic nonlinear optimal control problems with undiscounted\ncost functions is investigated while considering the errors existing in\napproximating respective functions. The boundedness of the results around the\noptimal solution is obtained based on quantities which are known in a general\noptimal control problem and assumptions which are verifiable. Moreover, since\nthe presence of the approximation errors leads to the deviation of the results\nfrom optimality, sufficient conditions for stability of the system operated by\nthe result obtained after a finite number of value iterations, along with an\nestimation of its region of attraction, are derived in terms of a calculable\nupper bound of the control approximation error. Finally, the process of\nimplementation of the method on an orbital maneuver problem is investigated\nthrough which the assumptions made in the theoretical developments are verified\nand the sufficient conditions are applied for guaranteeing stability and near\noptimality.\n