Non-neural Models Matter: A Re-evaluation of Neural Referring Expression Generation Systems

In recent years, neural models have often outperformed rule-based and classic\nMachine Learning approaches in NLG. These classic approaches are now often\ndisregarded, for example when new neural models are evaluated. We argue that\nthey should not be overlooked, since, for some tasks, well-designed non-neural\napproaches achieve better performance than neural ones. In this paper, the task\nof generating referring expressions in linguistic context is used as an\nexample. We examined two very different English datasets (WEBNLG and WSJ), and\nevaluated each algorithm using both automatic and human evaluations. Overall,\nthe results of these evaluations suggest that rule-based systems with simple\nrule sets achieve on-par or better performance on both datasets compared to\nstate-of-the-art neural REG systems. In the case of the more realistic dataset,\nWSJ, a machine learning-based system with well-designed linguistic features\nperformed best. We hope that our work can encourage researchers to consider\nnon-neural models in future.\n