ENCODING TEXTUAL INFORMATION FOR TEXT ANALYSIS

A computer-implemented method of encoding a word for use in a method of text analysis comprises receiving input text to be analysed, the input text comprising a first word which is not represented in a vocabulary set stored on a storage. The vocabulary set comprises a plurality of words and an associated word embedding vector for each word in the set. The method comprises identifying the first word as a word which is not represented in the vocabulary set and determining one or more sub-words within the first word with which to encode the first word. Each of the one or more sub-words corresponds with a word represented in the vocabulary set and having an embedding vector in the vocabulary set. The method comprises determining an encoding for the first word based on the one or more sub-words.