Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction

Traditionally, character-level transduction problems have been solved with\nfinite-state models designed to encode structural and linguistic knowledge of\nthe underlying process, whereas recent approaches rely on the power and\nflexibility of sequence-to-sequence models with attention. Focusing on the less\nexplored unsupervised learning scenario, we compare the two model classes side\nby side and find that they tend to make different types of errors even when\nachieving comparable performance. We analyze the distributions of different\nerror classes using two unsupervised tasks as testbeds: converting informally\nromanized text into the native script of its language (for Russian, Arabic, and\nKannada) and translating between a pair of closely related languages (Serbian\nand Bosnian). Finally, we investigate how combining finite-state and\nsequence-to-sequence models at decoding time affects the output quantitatively\nand qualitatively.\n