StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer

Text style transfer aims to controllably generate text with targeted\nstylistic changes while maintaining core meaning from the source sentence\nconstant. Many of the existing style transfer benchmarks primarily focus on\nindividual high-level semantic changes (e.g. positive to negative), which\nenable controllability at a high level but do not offer fine-grained control\ninvolving sentence structure, emphasis, and content of the sentence. In this\npaper, we introduce a large-scale benchmark, StylePTB, with (1) paired\nsentences undergoing 21 fine-grained stylistic changes spanning atomic lexical,\nsyntactic, semantic, and thematic transfers of text, as well as (2)\ncompositions of multiple transfers which allow modeling of fine-grained\nstylistic changes as building blocks for more complex, high-level transfers. By\nbenchmarking existing methods on StylePTB, we find that they struggle to model\nfine-grained changes and have an even more difficult time composing multiple\nstyles. As a result, StylePTB brings novel challenges that we hope will\nencourage future research in controllable text style transfer, compositional\nmodels, and learning disentangled representations. Solving these challenges\nwould present important steps towards controllable text generation.\n