Robust Learning of Recurrent Neural Networks in Presence of Exogenous Noise

Recurrent Neural networks (RNN) have shown promising potential for learning\ndynamics of sequential data. However, artificial neural networks are known to\nexhibit poor robustness in presence of input noise, where the sequential\narchitecture of RNNs exacerbates the problem. In this paper, we will use ideas\nfrom control and estimation theories to propose a tractable robustness analysis\nfor RNN models that are subject to input noise. The variance of the output of\nthe noisy system is adopted as a robustness measure to quantify the impact of\nnoise on learning. It is shown that the robustness measure can be estimated\nefficiently using linearization techniques. Using these results, we proposed a\nlearning method to enhance robustness of a RNN with respect to exogenous\nGaussian noise with known statistics. Our extensive simulations on benchmark\nproblems reveal that our proposed methodology significantly improves robustness\nof recurrent neural networks.\n