Classification via Tensor Decompositions of Echo State Networks

Typically when performing supervised classification tasks on data processed in an echo state network, the entire collection of hidden layer node states from the training dataset is shaped into a matrix, allowing one to use standard linear algebra techniques to train the output layer. However, the collection of hidden layer states is multidimensional in nature, and representing it as a matrix may lead to undesirable numerical conditions or loss of spatial and temporal correlations in the data. This work proposes a tensor-based supervised classification method on echo state network data that preserves and exploits the multidimensional nature of the hidden layer states of the reservoir. The method, which uses orthogonal Tucker-2 decompositions of tensors, is compared with the standard linear output weight approach in several numerical experiments on both synthetic and natural datasets, with results showing that the tensor-based approach tends to outperform the standard linear weights approach.