MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models

Large Language Models (LLMs) are distinguished by their massive parameter counts, which typically result in significant redundancy. This work introduces MaskLLM, a learnable pruning method that establishes Semi-structured (or ``N:M'') Sparsity in LLMs, aimed at reducing computational overhead during inference. Instead of developing a new importance criterion, MaskLLM explicitly models N:M patterns as a learnable distribution through Gumbel Softmax sampling. This approach facilitates end-to-end training on large-scale datasets and offers two notable advantages: 1) High-quality Masks - our method effectively scales to large datasets and learns accurate masks; 2) Transferability - the probabilistic modeling of mask distribution enables the transfer learning of sparsity across domains or tasks. We assessed MaskLLM using 2:4 sparsity on various LLMs, including LLaMA-2, Nemotron-4, and GPT-3, with sizes ranging from 843M to 15B parameters, and our empirical results show substantial improvements over state-of-the-art methods. For instance, leading approaches achieve a perplexity (PPL) of 10 or greater on Wikitext compared to the dense model's 5.12 PPL, but MaskLLM achieves a significantly lower 6.72 PPL solely by learning the masks with frozen weights. Furthermore, MaskLLM's learnable nature allows customized masks for lossless application of 2:4 sparsity to downstream tasks or domains. Code is available at https://github.com/NVlabs/MaskLLM.

Paper

Similar papers

Peer review

Reviewer qjsy8/10 · confidence 5/52024-06-16

Summary

This paper introduces the concept of learnable semi-structured sparsity (N:M sparsity). This extends to practical N:M sparsity to pretrained LLMs and makes the masks learnable. the paper also make the transfer of masks from other sparsity techniques to work for N:M. Finally, the general N:M masks learned can be easily fine-tuned for each task with great performance. the paper also shows extensive evaluations on various practical LLMs with the SOTA performance compared to other sparse LLM methods. ---------------------- The review will be short and does not reflect the time put in for the review or the quality of the paper. When the ideas are simple and clear -- I tend to write shorter reviews to the point.

Strengths

I really enjoyed reading this paper. It was a very practical paper on many levels I have read on sparsity and applicability to LLMs. The idea of differentiable masks is not new -- as pointed by the authors -- in CS (Savarese et al., 2019), STR (Kusupati et al., 2020) along with other methods mentioned in the paper. Same goes with N:M. However, bringing them together (again done earlier as mentioned by the paper) and combining them with pretrained LLMs makes is very useful in practice. I really like 2 other aspects apart from the base idea and practicality. THe transfer of one-shot pruning masks to N:M as scaffolds and then using learned N:M masks for downstream tasks was strong. The results are pretty solid as well. I also like the downstream evals and not just having perplexity values. the findings of the paper highlighted are very useful in further research. I am strongly in support of the paper unless I missed something obvious. I appreciate the author on the comprehensive paper.

Weaknesses

I do not find any glaring weaknesses concerns or questions about the current version of the paper. However, I might have missed something and would rely on the other reviewers if I was wrong. My only suggestion for the paper is to add a more comprehensive related work section to complete the paper. Maybe introducing learning sparsity be it mask learning with STE or STR would be a nice thing to add.

Questions

See above.

Rating

8

Confidence

5

Soundness

4

Presentation

4

Contribution

4

Limitations

Yes

Reviewer HusL8/10 · confidence 4/52024-07-11

Summary

This paper proposes MaskLLM, a learnable method to craft semi-structured sparsity in Large Language Models (LLMs). The approach involves modeling N:M masks with a categorical distribution, which is can be optimized through gumbel softmax. The key findings in this paper suggest that end-to-end learning of mask parameters can learn more effective and accurate sparsity patterns compared to one-shot methods. Additionally, the porposed MaskLLM also supports transfer learning for downstream tasks, where lossless masks can be learned for deployment.

Strengths

1. The ideas of learnable masks and transfer learning are innovative for Sparse LLMs. The proposed method enables task-oriented compression for downstream applications without necessitating the re-training of LLM weights, making it practical for real-world applications. 2. Results on several LLMs are positive. Table 1 demonstrates that the learnable mask method can achieve superior performance compared to state-of-the-art methods, while keeping the LLM weights frozen throughout the learning process. This indicates significant potential for further improvements in N:M sparsity within LLMs. 3. The results in Table 4 are interesting. They show that the learnable mask does not require so many training steps and samples to outperform the oneshot baseline. And the proposed method is more scalable with large-scale datasets. 4. The key ideas and findings in this paper are clear and well-organized. It is easy to understand the key messages in different experiments.

Weaknesses

1. One of my concerns is the use of non-public datasets and LLMs in this study. It would be beneficial for the paper to include more results using publicly available data and open-source models, such as LLaMA-3, to enhance the reproducibility and applicability of the findings. 2. This is a question about the storage cost in Table 6: Could the author clarify the actual storage costs, such as the file size on disk? Additionally, what is the data format used during the fine-tuning process? 3. Table 2 highlights the significance of the prior in mask selection. Does this suggest that the proposed method is only trying to fine-tune the prior mask rather than learn something new? What happens if we don’t have SparseGPT prior? Is the learned mask very similar to its prior?

Questions

Please refer to "Weaknesses".

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations are adequately addressed.

Reviewer 2tsd7/10 · confidence 4/52024-07-11

Summary

The paper introduces MaskLLM, a method for introducing semi-structured sparsity (N:M mask patterns) in LLMs. The authors show that existing model pruning methods such as SparseGPT result in significant loss in model quality at smaller scales (800M ~ 15B parameters) when using semi-structured methods. They formulate the problem of find a good N:M pattern as a mask selection problem from a candidate set of mask $S$. They formulate this further as a sampling problem, whereby they sample masks for all $\mathcal{W}^{1\times4}$ parameters in a layer to measure model quality - since exact mask computation is an intractable problem for large models. They propose using Gumbel-softmax sampling to figure out soft / differentiable masks that can learned through a training process, to optimize the sampling problem. The authors further find that learning of the masks by their proposed method can result in vanishing gradients throughout the network, and enable a sparse weight regularization to promote higher gradient norms through the network. They further find that using methods such as SparseGPT as mask priors enables more effective sampling (ie, learning) of the final masks. They follow these up with different experiments and ablations to highlight the strengths of their proposed method.

Strengths

1. The paper formulates the selection of N:M masks as an optimization problem, which enables scaling the method to large datasets and models. 2. The paper walks through the math required to understand their optimization formulation step-by-step, making it simple to understand. 3. The method focuses on common problems introduced by pruning (such as vanishing gradients of pruned weights etc.) and proposes ways to resolve them through regularization. 4. For each of the parts of the proposed method, the authors present ablations and results that validate their design choices. 5. The authors also showcase how their method can scale for transfer learning to downstream tasks (both via fine-tuning or via mask-transfer, as proposed via their prior initialized method) 6. The paper shows performance results from using their method (~1.3~1.4x faster) on A6000 GPUs.

Weaknesses

1. The best MaskLLM results presented rely on SparseGPT as a prior for mask intialization and then compare against SparseGPT for efficacy. This seems to be an unfair comparison, since you're using the method's best result and then improving on top of that. For e.g, without prior masks, the method seems to be much closer in performance to the SparseGPT method for the LLaMA-2 7B model (9.12 vs 10.42). It will be good to see how the method performs for say the 13B model without using any priors. 2. The authors show that SparseGPT has fundamental limits for improvement based on the number of samples (which is well documented and understood) - but the comparisons shown, for e.g., in Figure 4 are with different datasets? Did the authors test the collated datasets for the MaskLLM training with SparseGPT and what plateaus were observed there? 4. One inherent limitation to the method seems that there are many hyper-parameters to tune to find good masks ($\alpha$ for SparseGPT prior, $\mathcal{k}$ for logit scaling and $\tau$ for softmax temperature, $\lambda$ for the regularization coefficient). Do these hyper-parameters scale optimally for all model scales? Or is more fine-grained optimization needed as models scale up? - Also for the $\lambda$ parameter, the authors show that using $1e^{-4}$ results in highest gradient norm [Table 12], but the hyper-parameter used in Table 7 is $1e^{-5}$, can the authors clarify this aspect?

Questions

1. For the LLaMA-2 models, the authors mention using the official dataset from the paper. However, the paper has no mention of any datasets used for training the models. Can the authors clarify this discrepancy? 2. In Table 11, when the authors mention the score from RIA, does that include the channel permutation method enabled in the paper? 3. Can the authors verify their magnitude pruning results for downstream tasks for Llama2-7B (Table 1). For such a large perplexity difference between magnitude pruning and SparseGPT / Wanda - the downstream results seem too be high?

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

1. There are some inconsistencies in the links in the paper (for e.g., Appendix D mentioned in page 8 - for weight regularization maps to Section 4.1 in the paper - please fix this). 2. Throughout the paper, it is unclear what results map to what experiments. For example, for Figure 5, which models were used for the ablation? Same for the results in Tables 3, 5, and 13. Scanning the Appendix also did not help clarify this. It makes the results presented somewhat non-trivial to parse.

Reviewer AwSn7/10 · confidence 5/52024-07-14

Summary

The authors proposed a novel LLM pruning technique, by modeling the distribution of all possible masks and formulate the selection of optimal masks in a differentiable way.

Strengths

- Important and relevant problem setup. - The solution is novel. - Thorough evaluation across a range of model/dataset combinations.

Weaknesses

- Unclear what the computational cost of this technique is, and how that compares with the alternative/more straightforward technique of sparse pretraining/finetuning. - Unclear why the authors do not report speedup for all models/datasets. - Using perplexity as a proxy for downstream coding task performance is sub-optimal. Using benchmarks such as HumanEval is preferred.

Questions

- While this work is interesting and thoroughly executed already, can you comment on how this compares with more straightforward baseline of sparse finetuning/pretraining? What's the compute cost v.s. accuracy trade-off? - Finding 1. is vague in two ways: 1). unclear what "large scale dataset" mean, 1B tokens is not large scale for LLM, please just state the size of the dataset explicitly. 2). I don't think your experiments are enough to justify "Learnable Sparsity ... fully leverage computational resources to learn precise masks through end-to-end training". Just say your technique better leverages computational resources than prior art, which is what your experiments show. - What's the calibration set size to produce Table 1.? - For Table.2 SparseGPT, did you also used the weight updates in Learned Mask setup? - As for weight magnitude regularization (Finding 4), have you tried tuning the learning rate for downstream finetuning? This sounds like a learning rate issue.

Rating

7

Confidence

5

Soundness

3

Presentation

4

Contribution

3

Limitations

all addressed.

Reviewer qjsy2024-08-07

Thanks for the rebuttal. After looking at other reviews as well, I am in support of accepting this paper. Please ensure to update the related work section and incorporate the new experiments presented here in the final paper.

Authorsrebuttal2024-08-10

Thank you for the very positive comments and invaluable suggestions! We will incorporate all the mentioned results into the draft as suggested. Best regards, Authors of #1520

Reviewer 2tsd2024-08-08

Thank you for the detailed response to the reviews and additional experimental results. I understand that the 13B model results will take time, thank your for taking a stab at those. After reading all reviews and responses, I will update my rating to accept (score: 7). Please do incorporate the appropriate fixes / changes in the revised version of the paper.

Authorsrebuttal2024-08-08

Thank you very much for your encouraging feedback! We will ensure that all appropriate fixes and changes are incorporated into the revised version.

Authorsrebuttal2024-08-08

We would like to express our sincere gratitude for the insightful comments! We will improve the quality of our draft following the above suggestions.

Reviewer HusL2024-08-11

Response to Rebuttal

I have read through the response from the authors. The authors provided sufficient experiment results and statistics in the rebuttal to support their claim. My three concerns are adequately addressed. I have also read through other reviews that proves the work is rather solid. I will raise my score.

Authorsrebuttal2024-08-11

Thanks so much for the encouraging feedback! We will keep refining the draft with all the additional experiments. Best regards, Authors

Program Chairsdecision2024-09-25

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC