PDP: Parameter-free Differentiable Pruning is All You Need

DNN pruning is a popular way to reduce the size of a model, improve the inference latency, and minimize the power consumption on DNN accelerators. However, existing approaches might be too complex, expensive or ineffective to apply to a variety of vision/language tasks, DNN architectures and to honor structured pruning constraints. In this paper, we propose an efficient yet effective train-time pruning scheme, Parameter-free Differentiable Pruning (PDP), which offers state-of-the-art qualities in model size, accuracy, and training cost. PDP uses a dynamic function of weights during training to generate soft pruning masks for the weights in a parameter-free manner for a given pruning target. While differentiable, the simplicity and efficiency of PDP make it universal enough to deliver state-of-the-art random/structured/channel pruning results on various vision and natural language tasks. For example, for MobileNet-v1, PDP can achieve 68.2% top-1 ImageNet1k accuracy at 86.6% sparsity, which is 1.7% higher accuracy than those from the state-of-the-art algorithms. Also, PDP yields over 83.1% accuracy on Multi-Genre Natural Language Inference with 90% sparsity for BERT, while the next best from the existing techniques shows 81.5% accuracy. In addition, PDP can be applied to structured pruning, such as N:M pruning and channel pruning. For 1:4 structured pruning of ResNet18, PDP improved the top-1 ImageNet1k accuracy by over 3.6% over the state-of-the-art. For channel pruning of ResNet50, PDP reduced the top-1 ImageNet1k accuracy by 0.6% from the state-of-the-art.

Paper

Similar papers

Peer review

Reviewer v4x56/10 · confidence 4/52023-07-02

Summary

This paper proposes a parameter-free differentiable pruning, which uses a dynamic function of weights during training to generate soft pruning masks. It can generalize well on random/structured/channel pruning on both vision and NLP tasks. It achieves superior pruning results.

Strengths

1. The proposed PDP approach is novel and interesting, and can generalize well on random/structured/channel pruning. 2. The analysis in Section 3.1 is interesting and provides valuable insights. 3. The proposed PDP approach can be faster than existing differentiable pruning approaches, with smaller pruning accuracy loss.

Weaknesses

1. Instead of learning masks, which would introduce extra trainable parameters, PDP lets the weights of the network adjust themselves and generate soft masks. This approach may seem counterintuitive, as most weights are obtained and converged through expensive training processes. It may be more effective to adjust and learn masks rather than adjusting network weights to generate masks. 2. The paper only provides a simple explanation for why PDP is more effective. A theoretical proof may be more convincing. 3. Although the method is effective on the CNN and BERT models evaluated in this paper, its effectiveness on other models, such as larger vision transformers and LLMs, is still unclear. 4. An important related work[1] is missing. [1]Xia, Mengzhou, Zexuan Zhong, and Danqi Chen. "Structured pruning learns compact and accurate models." ACL (2022).

Questions

see the weakness section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The main limitation of the paper is that, aside from empirical experimental results, there is no theoretical proof or explanation as to why it is superior to existing non-parameter-free approaches. The solution presented in the paper raises doubts about its effectiveness on larger neural networks, such as ViT and LLMs.

Reviewer 8xNe7/10 · confidence 4/52023-07-03

Summary

This paper deals with the pruning algorithm PDP on DNN, the main innovation is to generate a differentiable mask based on weights using a designed threshold t, and the softmax function. This mask can ensure the gradient propagation during forward and backward process while training, reducing the accuracy loss.

Strengths

1. This paper is well written and easy to follow. 2. The proposed method is novel and provides an efficient way to learn pruning masks without incurring heavy parameter burden. 3. The performance increase remains constant on various sparse ratios and models.

Weaknesses

1. The employment of different masks can impact training process performance, yet a higher SAD[1] may result in a significant decline in accuracy. Given that the masks are still changeable in the later stages of PDP training, how can it provide high accuracy? [1] Learning N: M fine-grained structured sparse neural networks from scratch. In ICLR, 2020. 2. The description of PDP algorithm in section 3.2 may not be completely unambiguous. For the calculation of the conditional point t, the meaning of the paper should be to take the median of $r*n(W)$ and $(1-r)*n(W)$, but figure 3 (a) shows that $t=0.5\{min(W_h)+max(W_l)\}$, which seems to be wrong. 3. In Table 6, the performance of LNM seems to be quite different from that reported in the original paper. Please explain this.

Questions

Please see the weakness part.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

No limitations discussed.

Reviewer tv9j5/10 · confidence 4/52023-07-04

Summary

The paper focuses on DNN pruning and proposes a novel approach using a soft mask during training. The soft mask is designed to encourage weights around the pruning threshold to actively switch their states, aiding in the recovery of pruned weights. This method is simple, efficient, and introduces no additional parameters. Experimental evaluations of the proposed approach have been conducted on ResNets, MobileNets, and BERT. It also explores different pruning settings such as N:M pruning, random pruning and channel pruning.

Strengths

* The paper proposes a soft mask during training, which smooths the participation of weights around the pruning threshold in the neural network. This can give more chances for weights around pruning threshold to change their states, especially help pruned weights actively recover themselves. * The compared methods are enough in experiments. * The paper discusses relevant works and provides a comprehensive analysis.

Weaknesses

* Method: * Lack of differentiable analysis: The title and content of the paper stress the differentiable pruning, however, method part (3.2) does not give differentiable analysis. * Insufficient explanation of the design of "t" and calculation of "m(w)": Why is “t” in PDP training flow designed in this way? The paper lacks clear explanations about the design of “t” and calculation of “m(w)”. * Lack of analysis and ablation study on the efficacy of the proposed soft mask: I find that the paper adopts progressive pruning as illustrated in the algorithm chart in supplementary material. As progressive pruning is an effective technique for final accuracy, I am doubtful about the real effect of the proposed method. To better understand the effectiveness of the proposed soft mask, we need more analysis and ablation studies. * Experiments: * More experiments about Transformer architectures are preferred. The paper provides BERT models experiments on MNLI dataset. Experiments with ViT models or BERT models on the whole GLUE benchmark will be preferred. * In the main body, there are not enough sparsity ratio selections. Table 4 provides only one sparsity ratio for each model. More sparsity ratios, ranging from low to high are necessary to understand the effect of the proposed method. * Writing: * Redundancy between Figure 5 and Table 4: It seems that Figure 5 and Table 4 convey the same information. What is the difference between Figure 5 and Table 4 or why illustrate the same information twice? I think each table or figure should contain specific information, especially in the main body. * Improper organization of the paper: The paper first illustrates the benefits of PDP over existing pruning approaches, then introduces the PDP algorithm, which can be confusing. Before readers know the method clearly, it would be difficult to understand its benefits. The better solution would be to change their order or to reconsider the effects of section 3.1, such as serving it as a motivation section.

Questions

Please see the method and experiment points in the weakness part. If my concerns are addressed, I would like to increase my score.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

1 poor

Contribution

3 good

Limitations

There is no potential negative societal impact of their work.

Reviewer R2q97/10 · confidence 4/52023-07-06

Summary

This paper describes a novel pruning algorithm named parameter-free differentiable pruning (PDP). The core idea is to generate soft pruning masks (i.e., mask values are not finalized until the final iteration of fine-tuning) using a parameter-free, differentiable dynamic function of the weights of the network. This approach eliminates the need to train additional mask parameters (and related hyper-parameter tuning) and appears to be easy to integrate into existing training pipelines. PDP is evaluated on networks drawn from vision and NLP, and on both unstructured and structured N:M sparsity patterns. The paper reports both accuracy and MAC improvements and provides a comparison of PDP to other SOTA pruning algorithms.

Strengths

* The paper is fairly well-written, but could use some reorganization to help with the overall flow. * The paper targets the relevant and important problem of structured sparsification of large DNNs to improve inference efficiency. * While soft and progressive pruning are well-known in the literature, its parameter-free nature and the specific mask computation make PDP unique. * Evaluation methodology is sound, with relevant comparisons to other SOTA approaches, and results being reported on various types of networks (CNNs, attention-based, etc.) across disparate domains.

Weaknesses

Measuring inference efficiency in MACs can often be misleading. For instance, unstructured pruning can reduce MACs by getting rid of individual ineffectual computations, but this is difficult to realize in modern parallel hardware such as GPUs and TPUs. Reporting at least some data points with actual runtime numbers would be useful.

Questions

* What is the motivation for selecting the per-layer sparsity value (r) after a few epochs of training? How would a user select the right number of epochs for a given network, sparsity pattern and global sparsity degree? * Readability suggestion: consider moving Section 3.2 before 3.1.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Limitations have been adequately addressed in the paper.

Reviewer qvwL5/10 · confidence 2/52023-07-10

Summary

The paper introduces a new DNN pruning scheme called Parameter-free Differentiable Pruning (PDP), which is an efficient and effective train-time pruning method that offers state-of-the-art qualities in model size, accuracy, and training cost. Unlike existing pruning approaches, PDP generates soft pruning masks for weights in a parameter-free manner, making it easy to apply to various vision and natural language tasks, DNN architectures, and structured pruning constraints. The paper presents experimental results on several benchmark datasets, including MobileNet-v1 and BERT, demonstrating that PDP achieves impressive results in terms of model size, accuracy, and training cost.

Strengths

- The paper introduced a PDP, a novel differentiable pruning methods that is parameter-free, which uses a dynamic function of weights to generate soft pruning masks for the weights. - PDP can be applied tto structured and channel pruning, such as N:M pruning, where top-of-the-line GPUs support such configuration.

Weaknesses

- The PDP differentiable pruning does not introduce extra parameter, but it still need to generate (soft) mask from weight, which would induce extra activation maps, how is the memory consumption during differentiable pruning compare to other SoTAs? - The paper seem provide results on MobileNet-v1/v2 but not on MobileNet-v3, can the author elaborate on why?

Questions

As shown in weaknesses part.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The paper did not any (Peak) memory consumption result of PDP during differentiable pruning.

Reviewer tv9j2023-08-18

Post-Rebuttal

Thanks for the detailed response. I appreciate the author's explanation about Q1 and experiments. Thus I would increase the score to borderline accept. But I suggest the author to improve the writing and give more explanation like the response of Q1 in the paper.

Reviewer R2q92023-08-21

Thank you for responding to my questions. My score remains the same.

Reviewer v4x52023-08-21

Response to authors

Thanks for your further explanation, which answers my questions. I will increase the score.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC