Summary
This paper presents a method for explicit planning for LLMs for text generation:
1. It breaks text generation into “chunks” (or sentences”); and for generating the next chunk, it explicitly injects an “action” variable, obtained from the planner module, to condition the next chunk generation.
2. It trains a separate transformer model to predict a sequence of action variables, as the “plan” of the generation.
3. The action variable is obtained by clustering the sentence level embeddings for sentences in the training corpus, which means no explicit supervision.
Empirically, the authors find that the planning can somewhat improve generation quality (in terms of lower perplexity).
Reasons to accept
This paper presents an interesting idea that explicitly models the “planning” of the writing then uses it to guide/improve the generation. It automatically derives the planning in an unsupervised fashion using sentence-level embeddings of the training data. It separates the planning module and the language model, and the needed training is relatively lightweight. This idea is relevant to some of the previous work like sentence-level language modeling (Ippolito et al) and FUDGE (Yang and Klein).
Reasons to reject
This paper can be improved as follows:
Methodology:
1. The reference of “plan” / “action” is based on the sentence embeddings rather than explicit writing plans like “write a sentence about concept X”, “detail the characteristics of the city”. In essence, the planning module is doing sentence level language modeling. I would recommend the authors clarify that in the paper and I would not call the $a$ variables as “actions” but just next sentence embeddings.
2. Because the author uses clusters of semantic embeddings, the granularity for expressing the fine-grained actions for the next sentence might not be fine-grained enough to guide the next generation. For example, in table 3 and table 6, the authors show examples of the major clusters of the action vectors – it only reveals major semantic/topic differences like business/geograph. That fails to capture the fine-grained control of the details during the writing process – e.g., two consecutive sentences might talk about the same general topic but focus on different aspects/details. Hence, the usefulness of the planning module is actually unclear (which is also reflected in the empirical results).
3. While it’s good to decouple the planning module from the language models, currently the training of the planning module does not use any signal from the language models, which might make the plans less helpful. Alternatively, one can use sentence representation from the same language model to train the planning module (rather than sentence bert).
Experimental design and results:
1. Missing baselines: In table 1, the author only compares with a baseline that does not train the language model (the None column). At least the baseline where simply fine-tuning the language model on the same training data using a regular causal LM objective should be compared.
2. Perplexity might not be the best measurement, and one should compare the actual generation quality (In fact, improving perplexity from 27.89 to 26.69 might not necessarily indicate better generation quality). It would be helpful to directly compare generation quality, e.g., comparing rouge-2 of the generations, with/without using the plan.
3. Current table 1 evaluation (edit distance) is done at the sentence/chunk level, which might not be a realistic setup. It would be interesting to compare the quality of generating the complete article.
Questions to authors
There are some missing references:
FUDGE: Controlled Text Generation With Future Discriminators
For an in depth analysis of controlled generations, table 1 in Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo could be a helpful pointer.