Structural Adapters in Pretrained Language Models for AMR-to-text Generation

Pretrained language models (PLM) have recently advanced graph-to-text\ngeneration, where the input graph is linearized into a sequence and fed into\nthe PLM to obtain its representation. However, efficiently encoding the graph\nstructure in PLMs is challenging because such models were pretrained on natural\nlanguage, and modeling structured data may lead to catastrophic forgetting of\ndistributional knowledge. In this paper, we propose StructAdapt, an adapter\nmethod to encode graph structure into PLMs. Contrary to prior work, StructAdapt\neffectively models interactions among the nodes based on the graph\nconnectivity, only training graph structure-aware adapter parameters. In this\nway, we incorporate task-specific knowledge while maintaining the topological\nstructure of the graph. We empirically show the benefits of explicitly encoding\ngraph structure into PLMs using StructAdapt, outperforming the state of the art\non two AMR-to-text datasets, training only 5.1% of the PLM parameters.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC