Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation

Summary

This paper introduces an approach for dynamically pruning the FF of Transformer LLMs that doesn't require any extra training. The structured pruning is conditioned on the prompt. When computing the prompt the regular full model is used. Then, based on the prompt activations, only the top-k rows of each FF across all layers are being kept. The generation is performed with this pruned network. So essentially the pruning is dynamic at the sequence level. The authors present motivation for this approach by describing a behavior they call "flocking" where the activated neurons are mostly similar within a sequence but vary across different sequnces (studied on open source 7B and 13B models). They show that they can prune around 50% of FF for generation and mostly preserve classification and generation quality leading to about 1.25X speedup in LLama 13B.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

1. Simple but effective approach that doesn't require any extra training 2. Approach seem to work on several different LLMs and datasets 3. doesn't require any training so should be very easy to try on any open model

Reasons to reject

1. The writing and framing needs a bit improvement. The title says MoE but then the paper goes back and forth between MoE and pruning. The actual method is doing dynamic structured pruning. I don't think MoEs should be mentioned anywhere (maybe other than related work) and certainly not in the title since its confusing the reader. 2. Given that the method is actually pruning, I think the main weakness of the paper is lack of comparison to more pruning methods other than the presented naive magnitude approach. 3. The ablations part can also be improved. I am not sure what is the point of the sampling-based selection experiment. Instead, it will be interesting to see more ablations on the dynamic requirement for example if you take top-k when averaging activations over the full dataset and then keeping it static. 4. The method seems to work well on short generations but on longer ones the performance seems to substantially degrade. The authors suggest to use a long prompt for alleviating this but this defeats the efficiency purpose. 5. There are some more limitations to this method like having to select columns or rows for structural pruning rather than selecting from all neurons, and overall the speedup potential is limited with this pruning as even 50% parameter reduction of FF leads to small speedup as observed in experiments.

Questions to authors

1. Can you define exactly what you mean by relative FF activation? 2. It seems by the heatmaps that maybe the distribution is slightly different across layers. Have you tried using different k across layers?

Summary

The article introduces a new, training-free Mixture of Experts (MoE) method called GRIFFIN. This method capitalizes on a phenomenon observed in large language models (LLMs) known as "flocking," where specific feed-forward (FF) neurons exhibit consistently higher activation relative to others within sequences. GRIFFIN leverages these observations to selectively activate the most relevant FF neurons for efficient processing, thereby maintaining model performance while reducing computational demands.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

- GRIFFIN improves computational efficiency by reducing the number of active neurons during the model's forward pass, which lowers memory usage and speeds up processing without extensive hardware requirements. - Despite reducing the number of active parameters in FFN by up to 50%, GRIFFIN maintains near-original performance levels across multiple tasks, which is a significant achievement compared to other sparsity-inducing methods. - Unlike traditional MoEs that require retraining or fine-tuning, GRIFFIN operates without any additional training, making it easy to implement with pre-trained models.

Reasons to reject

Scalability Limitation Due to Batch-Size Constraint: GRIFFIN's current implementation is constrained to a batch size of one, which limits the model's ability to leverage parallel processing capabilities inherent in modern computational hardware. While a batch size of one achieves a modest speed-up of approximately 1.2x, this benefit does not scale linearly with larger batch sizes. This is due to the unique 'flocking' patterns observed in individual samples, which may vary significantly across different inputs within a batch. This variability makes it challenging to maintain consistent efficiency gains when processing multiple samples simultaneously. Such constraints on batch size could restrict the method's utility in real world model severing.

Summary

This paper discusses the phenomena of “flocking”, where FF activations in dense LLMs tend to concentrate by column within the same sequence, and proposes GRIFFIN, a novel technique that selects subsets of FFN parameters per sample based on flocking pattern on the prompt to achieve inference speed up without much loss of quality. The proposed method is training-free, empirically applicable to a list of LLMs, and simple to implement.

Rating

7

Confidence

3

Ethics flag

1

Reasons to accept

The algorithm this paper proposes is well-motivated and empirically strong. GRIFFIN tackles the important problem of efficient inference, achieving a decent boost in latency without costly retraining, and is empirically demonstrated to be widely applicable. This paper also highlights the flocking behavior and shows that it is possible to exploit such inherent structure for practical speed up. I can see this work spark future ideas along this topic.

Reasons to reject

I don’t find major reasons to reject the paper. Some lesser issues I have with the paper are 1. There is not much quantitative characterization of the flocking behavior. The paper shows many examples of flocking, but it’d be good to have some kind of quantitative measurement of this behavior at scale, to demonstrate how ubiquitous is the flocking behavior "in the wild". 2. ROUGE scores for Llama-13B and Mistral-7B on CNN-DM seems too low without any explanation (also Llama-13B < Llama 7B). This brings a bit doubt to the experiment results. It’d be good to provide some qualitative generation examples. 3. In tandem to #2 - it’d be great to provide generation samples with GRIFFIN, as ROUGE-based evals are n-gram-overlap based and may hide problems. It’d be good to rule out possibilities such as generating incoherent yet overlapping phrases, etc., by showing some qualitative samples. 4. A pedantic point: to posit GRIFFIN as an approach to convert a dense model to MoE seems a bit of a stretch of concept. GRIFFIN seems to have much more affinity to adaptive pruning or adaptive (structured) sparsity rather than MoE.

Questions to authors

1. By end of page 1 - “Perhaps an even bigger weakness of many of these methods is the inability to effectively carry over to pre-trained LLMs with non-ReLU activations.” - this is not evident to me, could you elaborate, or if this is from previous studies, cite the study? 2. Figure 3 - Rows of Z_bar are not unit vectors. Is this a mistake in the figure? 3. Table 2 - Llama 2 13B is significantly worse than 7B in CNN-DM, and 2.5 pts of ROUGE-1 is pretty much no overlap. Mistral 7B also sees this issue. What is happening for these experiments? What do the models generate? 4. The observation that flocking patterns are shared within 1 sequence and differ across sequences begs one question - what happens when you take 2 sequences that have distinct patterns and concat them together? will it create problems for GRIFFIN? the meta-question here is what causes patterns to stay stable within sequence, and will there be naturally occurring data that breaks this assumption (therefore potentially make GRIFFIN fail on them)? 5. Section 4.2 right before eqn 6 - “...we do this by aggregating taking…” → "...we do this by aggregating..."?

Summary

This paper introduces GRIFFIN, an interesting MOE method, which leverages special activation patterns in Feedforward layers for pruning. The key feature of GRIFFIN lies in its training-free design for selecting unique FF experts at the sequence levels. Besides, this method is adaptable to non-ReLU methods, which makes it practical in applications, given that many popular LLMs use other activations like SwiGLU. Experimental results demonstrate that, even with only 50% of FF parameters, an MOE LLM employing GRIFFIN can still maintain performance comparable to dense models.

Rating

5

Confidence

4

Ethics flag

1

Reasons to accept

1. The proposed method stands out for its training-free design, a practical advantage particularly beneficial for large-scale LLMs. The discovery of fixed patterns within Feedforward Neural Networks (FFNs) is an intriguing observation, evident across various LLM architectures such as LLaMA and Gemma, with activations like SwiGLU and GEGLU. This consistency underscores the potential compressibility of FFN layers. 2. GRINFFIN's simplicity is notable, as it relies solely on the L2-norm of activations during runtime. This streamlined approach not only enhances computational efficiency but also simplifies the implementation. Moreover, the selected experts, i.e., the pruned layers identified by GRINFFIN, can be seamlessly integrated into the decoding process for generating future tokens.

Reasons to reject

The main limitation of this paper is the absence of baseline methods in the experimental section. For instance, Dejavu[1] introduces a contextual structured sparsity approach to dynamically prune layers during runtime. This parallels the strategy adopted in this paper, which also focuses on adaptively pruning Feedforward Neural Networks (FFNs) for different inputs. However, the experiments presented in this submission solely involve magnitude pruning, which may not offer a robust baseline compared to more sophisticated methods like Dejavu. Integrating diverse baseline approaches could provide a more comprehensive assessment of the proposed method's performance and elucidate its relative strengths and weaknesses in comparison to existing techniques.

Questions to authors

Please refer to the "Reasons to Reject"

Thank you for the rebuttal

Thank for very much for the new result, and it looks quite promising. I will raise the score.

Thank you to the reviewers!

We thank all the reviewers again for making thorough comments and suggestions! Since the discussion period will come to a close soon, if not done so already, we would greatly appreciate if the reviewers could let us know if we have addressed their remarks, so we can further improve the quality of our paper. Thank you!