SEQUENCE GENERATION USING NEURAL NETWORKS WITH CONTINUOUS OUTPUTS

A machine learning system including a continuous embedding output layer is provided. Whereas traditional machine language translation or generation models utilize an output layer that include an single output for each word in the output vocabulary V, the present machine learning system includes a continuous embedding output layer that stores continuous vectors mapped to an m-dimensional vector space, where m is less than V. Accordingly, the present machine learning system processes an input string to produce an output vector and then searches for the continuous vector within the vector space that most closely corresponding to the output vector via, for example, a k-nearest neighbor algorithm. The system then outputs the output string corresponding to the determined continuous vector. The present system can be trained utilizing a cosine-based loss function.