Stay on topic with Classifier-Free Guidance

We have completed an analysis comparing CFG to SOTA Baselines, as you suggested. Thank you so much for the comments, would appreciate it if you could re-evaluate.

Dear Reviewer RjYY, Thank you so much for your review! We appreciate your insights, they have made this paper stronger. > The \gamma values in one context are poorly suited for another context, making CFG tricky in practice. We do find that gamma values transfer across models and across benchmark tasks, with values of ~2 being optimal for all LLaMA, Pythia, and GPT-2 models on all standards language model benchmarks we test (other than Winogrande, where CFG doesn't seem to help at all). This is shown in Fig. 5-7. We will be clearer about this, as a general recommendation to practitioners. > Have you explored this method in controllable NLG tasks or constraint-decoding tasks, or compared it with SOTA methods? We appreciate your suggestion. We have updated the paper with an experiment comparing CFG to one of the dominant baselines in controlled NLG, which is a classifier-guided text generation framework called FUDGE [1]. In short, we prompt GPT2 + FUDGE and GPT2 + CFG with 500 prompts, taken from the IMDB dataset [2] and measure the sentiment of the output. We run FUDGE with a sentiment classifier [a] for guidance and CFG with the following positive prompt: “That was a good movie!”. We evaluate the output with a separate sentiment classifier [b] and compare each against vanilla GPT2 outputs for the same prompt. (Both sentiment classifiers were reported to get held-out f1 > .9). FUDGE increases the sentiment of the output by .065 points (using the author-recommended control parameter, 1) whereas CFG increases the output by significantly more (0.312) using our default control strength, mentioned above. FUDGE faces other limitations: it is extremely slow. It takes over 100x more to generate each prompt, because at each step, classifier-guidance needs to be applied to potential candidate words. It requires a trained classifier. Other methods, like NADO [3], Gedi [4] and CTRL [5] additionally require expensive training, whereas CFG does not. Constrained-decoding tasks, like [6], are not quite appropriate in this setting. CFG makes no claim to restrict the output to single words. > Compared with text-to-image generation, the optimal \gamma value in the language modelling seems to be small (<2), while large \gamma value leads to poor performance. Have any observations on it? There are many reasons why text-to-image models might have higher \gamma values. In text-to-image generation the pixel range is (-1, 1), whereas the range for logits in language modeling is a lot larger. In text-to-image generation, the values are independent but in text-to-text there's a softmax, and thus changing the maximum logit value dramatically alters the whole distribution. The conditional and unconditional outputs may be more different in text-to-text than in text-to-image, leading to greater chances of output text degenerating. In text-to-image diffusion models, after a very small number of iterations, the differences between the conditional probability and the unconditional probability should be negligible, so a stronger strength might be required. We would be really appreciative if you were to consider these additional experiments, and if you are satisfied, to please consider raising the score on your review. Do you have any other comments or feedback? Thank you so much, again, as your comments have greatly improved the quality of this paper. [1] Yang, Kevin, and Dan Klein. "FUDGE: Controlled text generation with future discriminators." arXiv preprint arXiv:2104.05218 (2021). [2] https://huggingface.co/datasets/imdb [3] Meng, Tao, et al. "Controllable text generation with neurally-decomposed oracle." Advances in Neural Information Processing Systems 35 (2022): 28125-28139 [4] Krause, Ben, et al. "Gedi: Generative discriminator guided sequence generation." arXiv preprint arXiv:2009.06367 (2020). [5] Keskar, Nitish Shirish, et al. "Ctrl: A conditional transformer language model for controllable generation." arXiv preprint arXiv:1909.05858 (2019). [6] Lu, Ximing, et al. "Neurologic a* esque decoding: Constrained text generation with lookahead heuristics." arXiv preprint arXiv:2112.08726 (2021). [a] https://huggingface.co/stevhliu/my_awesome_model [b] https://huggingface.co/bhadresh-savani/distilbert-base-uncased-emotion

We have completed a memory-cost analysis, as you suggested. We would really appreciate it if you could look, and re-evaluate.

Hello 3Gz2, Thank you so much for your very thoughtful review, these points will help us improve the paper. We have made the formatting changes you've pointed out and will do another editing pass on the paper to make sure nothing slips through the cracks. We have completed experimental analyses, updating Section 4 and adding Appendix B.3 in a revised version of the paper — we would appreciate it if you examined our results and considered updating your score. We will summarize here: > A memory cost analysis is recommended. We have completed a memory analysis and will include our results in the paper. In general, we found a tradeoff between serving larger models, and serving a smaller model with CFG. The paper concludes that running a model with CFG provides comparable performance than a model with twice the parameter count. While it is clear that TRAINING a 50% smaller model is cheaper, you raise an important question about serving, and the memory impact wrt to sequence length / kv cache length. Should we serve with a model CFG or serve a vanilla model that is twice as big? Let’s call these M and M’, respectively. Let P be the parameter count of M (in floats), S the size of the kv cache, C the number of floats in the kv cache for a single token for M, and C’ the number of floats in the kv cache for a single token for M’. Inference on M occupies P + 2CS floats in memory Inference on M’ occupies 2P + C’S floats in memory The tradeoff favors CFG when VRAM is dominated by model parameters, while serving a bigger model would be optimal when the VRAM is dominated by the cache. After calculating `cache size per token = f(parameter count)`, we solve for S to find when both strategies use the same amount of VRAM, which delimits the optimal strategy change point. We end up with this chart: https://imgur.com/3nOdTqG.png For instance, if you have 10GB of VRAM, in order to maximise the VRAM available for the kv cache, models below 700M param count should be deployed without CFG. Models between 700M and 5B should be served with CFG. Models bigger than 5B can’t be served at all. (All VRAM calculations assume fp16 / bf16.) At a high level, this means that using CFG depends on your use-case. For researchers or small scale deployments where people are using the largest model that they can fit on their GPU, it's better to use CFG. However for very large scale commercial deployments, it makes more sense to increase the size of the model. We further note that increasing the size of the model is not always possible: OpenAI probably doesn't have a version of GPT-4 that's twice as big sitting around, and people using open models typically also have a maximum model size that's accessible to them. So once you wish to serve a model at least 50% as large as the maximum model size, the calculus favors CFG, regardless of VRAM. This reasoning has been added to the paper in Appendix B.3. > There is no guarantee that the Eq.6 will obtain a legal probability with the probabilities of all possibilities summing up to 1. This is a great point and we will be clearer in the paper. Our actual implementation of CFG is on the logits-level, see Equation (7). Because logits are transformed with a softmax function, we still have a valid sampling function. (Other established and popular methods alter sampling logits before softmax as well, e.g. nucleus sampling [1] and contrastive decoding [2]). Thank you again for your thoughtful comments. If you would consider raising your score in light of this response, we would really, really appreciate it! [1] Holtzman, Ari, et al. "The curious case of neural text degeneration." arXiv preprint arXiv:1904.09751 (2019). [2] Li, Xiang Lisa, et al. "Contrastive decoding: Open-ended text generation as optimization." arXiv preprint arXiv:2210.15097 (2022).