This paper is an attempt to bridge the gap between deep learning and\ngrammatical inference. Indeed, it provides an algorithm to extract a\n(stochastic) formal language from any recurrent neural network trained for\nlanguage modelling. In detail, the algorithm uses the already trained network\nas an oracle -- and thus does not require the access to the inner\nrepresentation of the black-box -- and applies a spectral approach to infer a\nweighted automaton.\n As weighted automata compute linear functions, they are computationally more\nefficient than neural networks and thus the nature of the approach is the one\nof knowledge distillation. We detail experiments on 62 data sets (both\nsynthetic and from real-world applications) that allow an in-depth study of the\nabilities of the proposed algorithm. The results show the WA we extract are\ngood approximations of the RNN, validating the approach. Moreover, we show how\nthe process provides interesting insights toward the behavior of RNN learned on\ndata, enlarging the scope of this work to the one of explainability of deep\nlearning models.\n