TECHNIQUES FOR PRUNING PHRASE TABLES FOR STATISTICAL MACHINE TRANSLATION

A computer-implemented technique includes receiving, at a server including one or more processors, a phrase table for statistical machine translation, the phrase table including a plurality of phrase pairs corresponding to one or more pairs of languages. The technique includes determining, at the server, a redundant set of phrase pairs from the plurality of phrase pairs and calculating first and second probabilities for each specific phrase pair of the redundant set. The second probability can be based on third probabilities for sub-phrases of each specific phrase pair. The technique includes determining, at the server, one or more selected phrase pairs based on whether a corresponding second probability for a specific phrase pair is within a probability threshold from its corresponding first probability. The technique also includes removing, at the server, the one or more selected phrase pairs from the phrase table to obtain a modified phrase table.