Recurrent Graph Tensor Networks: A Low-Complexity Framework for Modelling High-Dimensional Multi-Way Sequence

Recurrent Neural Networks (RNNs) are among the most successful machine\nlearning models for sequence modelling, but tend to suffer from an exponential\nincrease in the number of parameters when dealing with large multidimensional\ndata. To this end, we develop a multi-linear graph filter framework for\napproximating the modelling of hidden states in RNNs, which is embedded in a\ntensor network architecture to improve modelling power and reduce parameter\ncomplexity, resulting in a novel Recurrent Graph Tensor Network (RGTN). The\nproposed framework is validated through several multi-way sequence modelling\ntasks and benchmarked against traditional RNNs. By virtue of the domain aware\ninformation processing of graph filters and the expressive power of tensor\nnetworks, we show that the proposed RGTN is capable of not only out-performing\nstandard RNNs, but also mitigating the Curse of Dimensionality associated with\ntraditional RNNs, demonstrating superior properties in terms of performance and\ncomplexity.\n