Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers

The task of verbalization of RDF triples has known a growth in popularity due\nto the rising ubiquity of Knowledge Bases (KBs). The formalism of RDF triples\nis a simple and efficient way to store facts at a large scale. However, its\nabstract representation makes it difficult for humans to interpret. For this\npurpose, the WebNLG challenge aims at promoting automated RDF-to-text\ngeneration. We propose to leverage pre-trainings from augmented data with the\nTransformer model using a data augmentation strategy. Our experiment results\nshow a minimum relative increases of 3.73%, 126.05% and 88.16% in BLEU score\nfor seen categories, unseen entities and unseen categories respectively over\nthe standard training.\n