Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

Research on scaling large language models (LLMs) has primarily focused on model parameters and training data size, overlooking the role of vocabulary size. We investigate how vocabulary size impacts LLM scaling laws by training models ranging from 33M to 3B parameters on up to 500B characters with various vocabulary configurations. We propose three complementary approaches for predicting the compute-optimal vocabulary size: IsoFLOPs analysis, derivative estimation, and parametric fit of the loss function. Our approaches converge on the conclusion that the optimal vocabulary size depends on the compute budget, with larger models requiring larger vocabularies. Most LLMs, however, use insufficient vocabulary sizes. For example, we predict that the optimal vocabulary size of Llama2-70B should have been at least 216K, 7 times larger than its vocabulary of 32K. We validate our predictions empirically by training models with 3B parameters across different FLOPs budgets. Adopting our predicted optimal vocabulary size consistently improves downstream performance over commonly used vocabulary sizes. By increasing the vocabulary size from the conventional 32K to 43K, we improve performance on ARC-Challenge from 29.1 to 32.0 with the same 2.3e21 FLOPs. Our work highlights the importance of jointly considering tokenization and model scaling for efficient pre-training. The code and demo are available at https://github.com/sail-sg/scaling-with-vocab and https://hf.co/spaces/sail/scaling-with-vocab-demo.

Paper

References (80)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer zt866/10 · confidence 4/52024-06-24

Summary

The paper presents empirical scaling laws for the size of the vocabulary for LLMs. The findings in the paper are: - Empirically, the vocabulary size minimizing the loss increases when FLOPs are increased (Fig 2, right, Fig 3) - Through mathematical derivations from scaling laws, the optimal vocabulary size decreases when the embedding size increases. (Fig 4). - With a given FLOPs budget, 43k vocav size beats 32k on academic benchmarks like Boolq (Table 2.)

Strengths

- Vocabulary scaling laws are an interesting and novel research direction. - The paper is well written.

Weaknesses

- The results are probably mostly applicable to a small number of well-funded labs.

Questions

1. The abstract states that “beyond the conventional 32K” – is this really the convention? See e.g. GPT4o. 2. How does Table 2 look if you train on the same number of tokens instead of using the same FLOPs budget?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

na

Authorsrebuttal2024-08-11

Any New Comments Would be Greatly Appreciated

Dear Reviewer zt86, We are deeply grateful for your detailed review and the insightful suggestions you provided for our paper. We have carefully considered and responded to each of your comments in our rebuttal. As the Author-Review Discussion period is coming to an end, we want to ensure that all your concerns have been thoroughly addressed. If there are any remaining questions or issues, we would be glad to offer further clarification or make any necessary revisions.Thank you once again for your valuable feedback. Best regards, The Authors

Reviewer Y8zo6/10 · confidence 4/52024-07-12

Summary

This study primarily explores the role of vocabulary size in scaling large language models (LLMs). Traditional research has focused on model parameters and training data size, often overlooking the impact of vocabulary size. While intuitively larger vocabularies can enable more efficient tokenization by representing sentences with fewer tokens, they also increase the risk of under-fitting representations for rare tokens. By training models ranging from 33M to 3B parameters on up to 510B characters with various vocabulary configurations, we discovered that the optimal vocabulary size is constrained by the computational budget. We propose two methods to determine the optimal vocabulary size: an empirical IsoFLOPs approach and a fast derivative-based approach. Both methods indicate that vocabulary parameters should be scaled slower than non-vocabulary parameters. Nonetheless, vocabulary parameters are critical for performance and are under-allocated in current LLMs. By increasing the vocabulary size beyond the conventional 32K, we trained a better 3B parameter model despite using fewer training tokens. Our work reveals the underestimated role of vocabulary and the necessity of jointly considering vocabulary size, model parameters, and training data for efficient scaling.

Strengths

1. The study takes a holistic approach by examining the role of vocabulary size in the scaling of large language models (LLMs), addressing a gap in traditional research that often overlooks this aspect. 2. The introduction of two novel methods—an empirical IsoFLOPs approach and a fast derivative-based approach—for determining the optimal vocabulary size showcases the study's innovation and practical contributions. 3. Demonstrating that increasing vocabulary size beyond the conventional 32K can lead to better model performance with fewer training tokens highlights the practical implications and potential for efficiency gains.

Weaknesses

1. Lacks performance on large-scale models, such as whether increasing the vocabulary size to a greater extent performs better than existing models in the market. Table 2's experiments look a little bit less. 2. IsoFLOPs method is very sensitive, the experiments also looks not enough.

Questions

In Use Case section, the authors mentioned that (2) One has already conducted scaling law experiments following the Chinchilla laws with a fixed vocabulary size (e.g., 244 32K) and aims to estimate the optimal vocabulary size for a given model parameter. In determining the scaling law for the relationship between non-embedding model size and data (such as the Chinchilla law), why is it assumed that the vocabulary size is independent of these two factors

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the authors addressed limitations.

Authorsrebuttal2024-08-06

Response to reviewer Y8zo's second part

### W2.2: The experiments also looks not enough. It is noteworthy that we conduct extensive experiments on **1200 models pre-trained from scratch** (6 non-vocabulary parameters settings x 10 vocabulary sizes settings x 20 training data settings) for the fitting of our vocabulary scaling law. The key contributions in this paper is the several findings about how the vocabulary affects the model performance and how much compute should be allocated on the vocabulary based on the proposed 2 approaches. Following the previous study [1,2,3], we mainly use the held-out validation loss value for the evaluation of the trained 1200 models. It is a better metric than the downstream tasks performance as the held-out loss provides an unbiased measure of the model’s ability to generalize to new data, but also enjoys high computing efficiency. Instead, the performance of downstream tasks has a great variety across different tasks, which is not suitable as the main evaluation metric. The evaluation of downstream tasks is part of the ways to verify our prediction, therefore we do not take too much content to discuss it in our main paper. For downstream tasks, we conduct more experiments in the answer of your #Q1. The new results will be added in our polished version. ### Q1: In determining the scaling law for the relationship between non-embedding model size and data (such as the Chinchilla law), why is it assumed that the vocabulary size is independent of these two factors. Thanks for your question! We do not assume that the vocabulary size is independent of parameters and data. Instead, we make some adjustments in the Section of Preliminary: 1) We break down the total parameters into non-vocabulary parameters and vocabulary parameters; 2) We measure data not in tokens but in training characters. By doing so, the vocabulary size $V$ is independent with the non-vocabulary parameters $N_{nv}$ and the number of training characters $H$. In an experimental configuration, the developers can vary the vocabulary size without affecting non-vocabulary parameters or training characters. Then, we details our motivation why we separate the vocabulary parameter and non-vocabulary parameter below: Traditionally, scaling up model parameters in language models has been approached in two ways: increasing depth (i.e., the number of layers) or width (i.e., the hidden size). Current empirical practices often involve expanding both simultaneously [4]. This approach overlook crucial distinctions in how different parameters benefit from parameters expansions. Non-vocabulary parameters can benefit from increases in both depth and width, allowing for more complex hierarchical representations and broader feature capture. In contrast, vocabulary parameters, associated with word embeddings and language model heads, are generally confined to a single layer, limiting their ability to benefit from increases in the model depth. This disparity in growth potential between non-vocabulary and vocabulary parameters suggests that to maintain a balanced growth rate, it is better to separate the vocabulary parameter and non-vocabulary parameter into consideration. [4] Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Tran, Dani Yogatama, and Donald Metzler. 2023. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12342–12364, Singapore. Association for Computational Linguistic.

Authorsrebuttal2024-08-11

Any New Comments Would be Greatly Appreciated

Dear Reviewer Y8zo We sincerely appreciate your thorough review and the valuable suggestions and comments you provided for our paper. We have carefully considered each of your points and have addressed them in detail in our rebuttal. With the Author-Review Discussion period nearing its conclusion, we want to ensure that all your concerns have been fully addressed. If there are any questions or unresolved issues, we are eager to provide further clarification or make any necessary revisions. Thank you again for your thoughtful feedback. Best regards, The Authors

Reviewer EsaU7/10 · confidence 2/52024-07-18

Summary

This paper investigates the impact of vocabulary size on the efficiency of large language models (LLMs). Using models with 33 million to 3 billion parameters, it finds that optimal vocabulary size is limited by computational resources. The study introduces two methods to determine the best vocabulary size, showing that vocabulary parameters should scale slower than other parameters. Results highlight the significant yet underestimated role of vocabulary in scaling LLMs effectively, suggesting that larger vocabularies can improve model performance.

Strengths

The paper addresses a unique aspect of language model scaling by investigating the impact of vocabulary size on model performance, a dimension that is often overlooked in LLM research. The introduction of two novel methods to determine the optimal vocabulary size—empirical IsoFLOPs and a derivative-based approach—provides practical tools for optimizing LLM training and deployment. The analysis in this paper is quite in-depth, and some conclusions can provide references for subsequent LLM training efforts.

Weaknesses

This paper conducted experiments on language models of various parameter sizes, but the largest model tested was only 3 billion parameters. It would be better if we could further verify models with more than 7 billion parameters. I believe both the industrial and academic communities are eager to know whether the scaling law for vocabulary can generalize to larger models.

Questions

NA

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

This paper discusses the limitations in Appendix B.

Authorsrebuttal2024-08-12

Any New Comments Would be Greatly Appreciated

Dear Reviewer EsaU, We are truly appreciative of the time and effort you have dedicated to reviewing our paper. Your thoughtful feedback and constructive suggestions are valuable to us. We have carefully addressed your comments in our rebuttal to enhance the quality of our work. As we approach the final days of the Author-Review Discussion period, we would like to ensure that all your concerns have been comprehensively addressed. Should there be any remaining questions or issues, we are willing to provide further clarification or additional revisions.Thank you once again for your insightful contributions to our work. Best regards, The Authors

Reviewer Y8zo2024-08-12

Thank you for your thorough response to my questions. I am pleased to see the new results. However, there is room for improvement in this paper, such as Figure 3 (left), where the distribution of model sizes is very uneven. For example, for models below 100M, the basic points of model size are concentrated at 33M and ~90M, which will seriously affect the exponential factor of the fitted curve.

Authorsrebuttal2024-08-12

Response to reviewer Y8zo

Dear Reviewer Y8zo, Thank you for your timely feedback. We're pleased that you found our new results compelling and appreciate your insightful comments on areas for improvement, particularly regarding Figure 3 (left). We would like to offer the following clarifications: Our study expands on traditional scaling law approaches by incorporating additional parameters: Nnv (non-vocabulary parameters), Nv (vocabulary parameters), and H (training characters). This contrasts with previous work that primarily considered N (total model parameters) and D (training tokens). The inclusion of these additional factors adds complexity to the scaling law fitting process. Given the high computational costs associated with scaling law experiments, we concentrated our efforts on a limited set of non-vocabulary parameters while exploring a broader range of vocabulary parameters. This focus aligns with the primary objective of our research: investigating the impact of vocabulary size on model performance. To achieve this, we deliberately selected 10 groups with varying vocabulary sizes for each of the 6 fixed groups of non-vocabulary parameters. While this design choice accounts for the uneven distribution seen in Figure 3 (left), it also allows for a more thorough exploration of vocabulary size effects, as demonstrated by the diverse data points in Figure 3 (middle). Our experimental design and results robustly support our main conclusion: larger models benefit from larger vocabularies. We believe these clarifications provide a more comprehensive understanding of our research methodology and findings. We appreciate your careful review and hope these explanations enhance your confidence in our paper. We look forward to any further feedback you may have. Best regards, The Authors

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC