Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality

Prompt-based continual learning is an emerging direction in leveraging pre-trained knowledge for downstream continual learning, and has almost reached the performance pinnacle under supervised pre-training. However, our empirical research reveals that the current strategies fall short of their full potential under the more realistic self-supervised pre-training, which is essential for handling vast quantities of unlabeled data in practice. This is largely due to the difficulty of task-specific knowledge being incorporated into instructed representations via prompt parameters and predicted by uninstructed representations at test time. To overcome the exposed sub-optimality, we conduct a theoretical analysis of the continual learning objective in the context of pre-training, and decompose it into hierarchical components: within-task prediction, task-identity inference, and task-adaptive prediction. Following these empirical and theoretical insights, we propose Hierarchical Decomposition (HiDe-)Prompt, an innovative approach that explicitly optimizes the hierarchical components with an ensemble of task-specific prompts and statistics of both uninstructed and instructed representations, further with the coordination of a contrastive regularization strategy. Our extensive experiments demonstrate the superior performance of HiDe-Prompt and its robustness to pre-training paradigms in continual learning (e.g., up to 15.01% and 9.61% lead on Split CIFAR-100 and Split ImageNet-R, respectively). Our code is available at \url{https://github.com/thu-ml/HiDe-Prompt}.

Paper

Similar papers

Peer review

Reviewer UkQY5/10 · confidence 4/52023-07-03

Summary

This paper introduces a CIL approach that specifically addresses the use of pre-trained models. It is intriguing and significant to explore the most suitable CIL method for pre-trained models.

Strengths

1. CIL and how to do CIL in pre-trained models are very important 2. The writing is smooth and very easy to follow

Weaknesses

Excluding the assumption of a pre-trained model, this paper reduces to the existing research (as mentioned in line 233). Thus, I recommend conducting further analysis on CL for pre-trained models. 1. Considering the extensive exploration of pre-trained models in NLP, it is crucial for this paper to compare its approach with NLP continual learning baselines, e.g., [1,2,3]. 2. I see no reason why the theory should be restricted to prompts alone. Other parameter-efficient tuning methods such as adapters, Lora, and Prefix can easily be utilized. The author should include a comparison with these methods (see 1 references). 3. In addition, I suggest comparing this paper's approach with TIL methods as well, which do not require TII. This would serve as an additional evaluation to assess the proposed method's performance in WTP and TAP. [1]: Achieving forgetting prevention and knowledge transfer in continual learning, NeurIPS 2021 [2]: Continual Pre-training of Language Models, ICLR 2023 [3]: Continual Learning of Natural Language Processing Tasks: A Survey. https://arxiv.org/abs/2211.12701

Questions

See above

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

I think the discussion of limitations in Sec. 6 makes sense.

Reviewer 6q9Z7/10 · confidence 5/52023-07-04

Summary

This work provides a comprehensive analysis of state-of-the-art prompt-based approaches for continual learning with the use of pre-training. The authors empirically demonstrate a clear performance degradation of current strategies under realistic self-supervised pre-training and extensively analyze the exposed sub-optimality. The authors then provide an in-depth theoretical analysis of the continual learning objective in the context of pre-training, which can be decomposed into three hierarchical components, and propose an innovative approach to optimize them explicitly. Extensive experiments on various pre-training paradigms have demonstrated the clear advantages of the proposed method.

Strengths

1. This paper provides a well-organized formulation of state-of-the-art prompt-based approaches, assessing them as a unified perspective. 2. The empirical analysis reveals the degraded performance of prompt-based approaches under self-supervised pre-training, which is an important issue for practical applications. 3. The theoretical analysis is very interesting. The hierarchical components are specific to continual learning in the context of pre-training and apply to the settings of task-/domain-/class-incremental learning. 4. The proposed method allows for explicit optimization of the continual learning objective through adaptively leveraging pre-training and prompt architectures. Experimental results demonstrate a significant improvement in continual learning performance under different pre-training paradigms.

Weaknesses

1. Based on proofs in supplementary materials, is the notation $\bar{c}$ in Eq.(8) equal to $y$? Please check it. 2. Could the authors further discuss some concurrent related work such as PromptFusion [1] that exploits prompt and FSA [2] that exploits FiLM adapters for continual learning with pre-training. [1] PromptFusion: Decoupling Stability and Plasticity for Continual Learning. arXiv preprint arXiv:2303.07223. [2] First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning. arXiv preprint arXiv:2303.13199. 3. Although experimental results have been sufficient enough, it would be better if some fine-grained datasets can be further analyzed, such as CUB.

Questions

Some minor concerns remain to be addressed. Please refer to the Weakness.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors have properly discussed the limitations and potential negative societal impacts.

Reviewer Qf6e5/10 · confidence 3/52023-07-06

Summary

This paper studies application of prompts in pre-trained models for continual learning. Building upon the problem of [1], the authors introduce the Task-Adaptive Prediction (TAP) for the CIL problem using pre-trained networks. They demonstrate that a good TAP, WTP, and TII performances are necessary and sufficient for a good CIL model. The authors emphasize the significance of these factors, especially in the context of continual learning with pre-trained self-supervised model. [1] Kim et al., Theoretical study on solving continual learning. NeurIPS 2022

Strengths

- The paper is easy to follow and the proposed approach is well motivated - The paper has provided theoretical insights - The proposed method is much stronger than the existing baselines

Weaknesses

- The method heavily relies on pre-trained models trained with ImageNet. This can be an issue, especially for supervised pre-training, since ImageNet already contains classes similar or identical to classes used for CL and there could be information leak from the pre-training classes to continual learning classes [2]. - The theoretical contribution is somewhat ambiguous since the decomposition and theorem 1 and 2 are similar to [1]. [2] Kim et al., A multi-head model for continual learning via out-of-distribution replay. CoLLAs, 2022.

Questions

- I couldn’t fully-understand why it’s necessary to have TAP in the CL problem using pre-trained networks, considering that it wasn’t required when training from scratch [1]. - I am curious about the performance of the proposed method when the feature extractor is pre-trained using only the classes that are dissimilar to CL classes as [2] - Is the theoretical analysis only relevant to the prompt-based method?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Refer to Weaknesses and Questions

Reviewer tgkY7/10 · confidence 5/52023-07-08

Summary

The authors provide strong empirical analysis on existing "prompting for continual learning" papers. They propose a new hierarchical prompting method that includes several components which take advantage of unstructured data representations. The authors not only propose an interesting approach with SOTA performance, but they have very detailed experiments using several pre-training backbones and datasets.

Strengths

In general, I feel that for this subject area (which I am sure will have many neurips submissions), this paper is overall high quality. 1) I appreciate another "prompting for continual learning" paper that starts with insights and findings into existing methods, laying an intuitive foundation for the proposed approach. 2) With good intuitive and theoretical backing that seems to reasonable, the method has great performance gains over SOTA. 3) Nice, detailed analysis and ablation study. I checked carefully to see if maybe there was a "hidden" component that does most of the heavy lifting of the method, but the contribution components seem to all be impactful in some way. 4) I am incredibly thankful to see the experiments conducted on several different backbones. This code and framework will be a fantastic contribution on its own, along with the analysis, before even considering the method. As someone who has experience in this problem setting, I think open-source code for this project will be greatly appreciated.

Weaknesses

1) Task-id prediction for class-incremental learning on random splits of classes (i.e., where there is no actual task structure), seems a red flag to me that the pre-trained backbone is too strong for the proposed task datasets, and that it is actually taking advantage of the unfair backbone in order to "circumvent" class-incremental learning, instead posing it as the much easier task-incremental learning problem. However, this weakness also exists in the competing methods, and the strengths of the paper out-weigh this weakness imo. 2) Figure 2 c and d is very confusing. Which method is this for? If you are making strong claims about task specificity, should it not be done for all methods? 3) This is not relevant to your findings, and I am not requesting any experiments, but Coda-prompt might perform better at a lower constant learning rate if the high, constant learning rate is hurting its performance. Learning rate decay is also not a "trick", either - it is a valid hyperparameter choice and important for many representation learning methods.

Questions

a) See weakness 2 b) what is the computation cost of your methods compared to the baselines? I feel that even with a strong training time cost, I am happy to vote for paper acceptance. Just want to ensure transparency.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Discussed reasonable limitations - there is no obvious potential negative societal impact

Reviewer pdqM8/10 · confidence 5/52023-07-09

Summary

The paper suggests a new prompt-based continual learning method by leveraging combinatorial objectives with *within task prediction*, *task-identity inference*, and *task-adaptive prediction*. The paper first summarizes recent prompt-based continual learning techniques and demonstrates their unstable performance according to the pre-trained backbones (particularly, initializing with self-supervised representation degrades their performance significantly). And then, proposes a new regularization technique containing core components, including ensemble prompting, contrastive regularization, and TAP loss.

Strengths

The paper provides a comprehensive analysis and theoretical proof of the motivation. And the suggested idea is reasonable, and the methodological design is also clear. In experiments, the proposed method consistently surpasses strong baselines in terms of conventionally used continual learning metrics under multiple pre-trained backbones, and its improvement and stability are a bit impressive. Further, they provide the necessary ablation study and analysis in the main paper and appendix.

Weaknesses

The effect of the prompt ensemble in the method is not discussed. And the different behavior and impact of un-/instructed representation for continual learning are partially discussed in Figure 5, but I recommend a more comprehensive and detailed discussion/analysis.

Questions

N/A

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Please see the weakness.

Reviewer pdqM2023-08-14

Reviewer response

Thank you so much! After sincerely reading the author's rebuttal, all concerns are solved! I'm happy to keep my initial score. Congratulation on publishing strong work in the continual learning field!

Authorsrebuttal2023-08-14

Thank you

We thank the reviewer for finding our response satisfactory and are happy to know the very positive rating. Best, Authors

Authorsrebuttal2023-08-14

Look forward to further feedback

We thank you again for the valuable and constructive comments. We hope you may find our response satisfactory and raise your rating accordingly. We are looking forward to hearing from you about any further feedback. Best, Authors

Reviewer UkQY2023-08-19

I appreciate the authors' response and have also read the reviews from other reviewers. In fact, I share a similar concern as reviewer Qf6e. Specifically, I'm unclear about the theoretical basis for the impact of pre-training. To me, the difference between pre-training or not lies in the initialization, and I do not fully understand how this could lead to a theoretical difference.

Authorsrebuttal2023-08-20

Author response

Thank you for your feedback. The objective of continual learning is to perform well on sequentially arrived tasks. There must be a theoretical basis for the huge performance differences between pre-trained and randomly-initialized weights in continual learning, especially for rehearsal-free CIL, which provides the initial motivation of our theoretical study. An effective use of pre-trained knowledge for downstream continual learning also depends on specialized theories and methodologies. Specifically, the randomly-initialized parameter set $\theta_0$ carries no information and is greatly different from the optimal solution. It needs to be substantially changed in continual learning and should not be took into account in theoretical analysis of the objective. In contrast, the pre-trained parameter set $\theta$ carries beneficial pre-trained knowledge for downstream tasks. If the pre-training is adequately strong, $\theta$ is already close to the optimal solution and only requires appropriate fine-tuning. Therefore, $\theta$ needs to be stabilized (usually frozen) in continual learning and should be considered in theoretical analysis. (An intuitive summary: the random initialization $\theta_0$ is overwritten, while the pre-trained initialization $\theta$ is preserved.) In our case, without considering the impact of pre-training, TAP = TII * WTP, due to classifying the same input in the same semantic space, i.e., $\frac{\exp(h{\psi}(f(\boldsymbol{x}))[y])}{\sum_{j=1}^{t}\sum_{y' \in \mathcal{Y}j} \exp(h{\psi}(f(\boldsymbol{x}))[y'])}=\frac{\exp(\hat{h}{\omega}(f(\boldsymbol{x})[\bar{i}])}{\sum_{i=1}^{t} \exp(\hat{h}{\omega}(f(\boldsymbol{x})[i])}*\frac{\exp(h{\psi}(f(\boldsymbol{x}))[\bar{j}])}{\sum_{j \in \mathcal{Y}i} \exp(h_{\psi}(f(\boldsymbol{x}))[j])}$. With considering the impact of pre-training (the introduction of $\theta$), TAP != TII * WTP, because TII * WTP is to calculate $\frac{\exp(\hat{h}{\omega}(f(\boldsymbol{x})[\bar{i}])}{\sum_{i=1}^{t} \exp(\hat{h}{\omega}(f(\boldsymbol{x})[i])}$ $*\frac{\exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[\bar{j}])}{\sum_{j \in \mathcal{Y}i} \exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[j])}$, while TAP is to calculate $\frac{\exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[y])}{\sum_{j=1}^{t}\sum_{y' \in \mathcal{Y}j} \exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[y'])}$ (please refer to our latest response to Reviewer Qf6e for more details). In other words, the backbone parameters are frozen to stabilize pre-trained knowledge, with additional prompts $\boldsymbol{p}{i}$ to fine-tune the semantic space (i.e., representations). TAP is essentially classifying the instructed representation ($\boldsymbol{h} \in \mathcal{H}{i}^c$ in Eq. (12)), while TII * WTP is performed on uninstructed representation ($\hat{\boldsymbol{h}} \in \hat{\mathcal{H}}{i}^{c}$ in Eq. (11)). Targeting on different semantic spaces, TAP and TII * WTP are clearly different, i.e., TAP != TII * WTP. Strong empirical results also demonstrate their respective contributions to continual learning performance (see ablation study in Table 2). We thank you again for the valuable feedback. We will incorporate the above discussion in the final version to make our theoretical analysis clearer. If you have any further questions, please let us know.

Authorsrebuttal2023-08-14

Look forward to further feedback

We thank you again for the valuable and constructive comments. We hope you may find our response satisfactory and raise your rating accordingly. We are looking forward to hearing from you about any further feedback. Best, Authors

Reviewer Qf6e2023-08-18

Response to the author comments

Thank you for providing detailed responses. However, I still do not fully understand the theoretical part. With current manuscript, I don't see a significant theoretical contribution of this paper. The outcomes seem to be a direct extension of those presented in [1] with one additional consideration, $max\\{CIL, TAP\\}$. Moreover, I have noticed that the notations, necessary and sufficient conditions, theorem 1, and proofs are very similar to [1]. Regarding the author's comment A2, I don't see the distinction made by having the $\theta$ condition in the probability. The probability decomposition as well as all the theoretical results in [1] remain the same with or without $\theta$. Regarding the author's comment A3, what is the difference between the TAP probability $P(\mathcal{X}^y | D, \theta)$ and CIL probability $P(\mathcal{X}_{i, j} | D, \theta)$? Why TAP = TII * WTP in [1] becomes TAP != TII * WTP in this pre-training setting? Misc. Figure 3 is presented, but it is not referenced in the paper. I am looking forward to your comment!

Authorsrebuttal2023-08-19

Author response (part I)

Thanks for your feedback. Here we explain your additional questions about our theoretical part as below (we reorganize the order of questions and answers for clarity). **Q2-1: With current manuscript, I don't see a significant theoretical contribution of this paper. The outcomes seem to be a direct extension of those presented in [1] with one additional consideration, max[CIL,TAP]. Moreover, I have noticed that the notations, necessary and sufficient conditions, theorem 1, and proofs are very similar to [1].** A2-1: Our main contribution in the theoretical part is to demonstrate the importance of WTP, TII and **TAP** for continual learning with **pre-training**. This is distinguished from the main conclusion in [1] due to the introduction of pre-trained knowledge, with the necessity of all three components empirically justified in Table 2. As the additional consideration, TAP is performed specifically on the **instructed representation space**, which is highly dependent on the pre-training backbone (i.e., $\theta$), aiming to improve predictions by coupling it with TII and WTP. Since [1] is a pioneering theoretical study decomposing the objective of continual learning from scratch, we build on its notation and analysis for the shared part (i.e., WTP and TII), in order to provide an explicit comparison and demonstrate clearly our own contribution (i.e., TAP in response to the impact of pre-training). In fact, the differences and connections of [1] and ours have been explicitly discussed in our paper (see line 231-234). As summarized in the contribution part of the paper (line 50-56), our theoretical results provide strong insights to analyze the sub-optimality of representative methods for prompt-based continual learning and to address the sub-optimality with an innovative approach. Therefore, we respectfully argue that our theoretical contribution is not in isolation but highly synergistic with the empirical findings and performance improvements, as acknowledged by Reviewer pdqM, 6q9Z and tgkY. These results can potentially provide a unified perspective for parameter efficient fine-tuning in continual learning with pre-training, as shown in the response to your Q5 in the Rebuttal page. **Q2-2: Regarding the author's comment A3, what is the difference between the TAP probability and CIL probability? Why TAP = TII * WTP in [1] becomes TAP != TII * WTP in this pre-training setting?** A2-2: In this paper, the TAP probability $ P(\boldsymbol{x} \in \mathcal{X}^{y}|\mathcal{D},\theta)$ is a categorical distribution over all observed classes $\bigcup_{k=1}^{t} \mathcal{Y}k$. The typical CIL probability $P(\boldsymbol{x} \in \mathcal{X}{\bar{i},\bar{j}}|\mathcal{D},\theta)$ can be further divided into TII * WTP where $\theta$ carries the pre-trained knowledge. Without considering pre-training as in [1], TAP = TII * WTP, i.e., $ P(\boldsymbol{x} \in \mathcal{X}^{y}|\mathcal{D})=P(\boldsymbol{x} \in \mathcal{X}{\bar{i},\bar{j}}|\mathcal{D})$, due to classifying the same input in the same semantic space, i.e., $\frac{\exp(h{\psi}(f(\boldsymbol{x}))[y])}{\sum_{j=1}^{t}\sum_{y' \in \mathcal{Y}j} \exp(h{\psi}(f(\boldsymbol{x}))[y'])}=\frac{\exp(\hat{h}{\omega}(f(\boldsymbol{x})[\bar{i}])}{\sum_{i=1}^{t} \exp(\hat{h}{\omega}(f(\boldsymbol{x})[i])}*\frac{\exp(h{\psi}(f(\boldsymbol{x}))[\bar{j}])}{\sum_{j \in \mathcal{Y}i} \exp(h_{\psi}(f(\boldsymbol{x}))[j])}$. In contrast, with considering pre-training the network $f$ with parameter set $\theta$ in our paper, TAP != TII * WTP, i.e., $ P(\boldsymbol{x} \in \mathcal{X}^{y}|\mathcal{D},\theta)\neq P(\boldsymbol{x} \in \mathcal{X}{\bar{i},\bar{j}}|\mathcal{D},\theta)$. This is because, TII * WTP is to calculate $\frac{\exp(\hat{h}{\omega}(f(\boldsymbol{x})[\bar{i}])}{\sum_{i=1}^{t} \exp(\hat{h}{\omega}(f(\boldsymbol{x})[i])}$ $*\frac{\exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[\bar{j}])}{\sum_{j \in \mathcal{Y}i} \exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[j])}$, while TAP is to calculate $\frac{\exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[y])}{\sum_{j=1}^{t}\sum_{y' \in \mathcal{Y}j} \exp(h{\psi}(f(\boldsymbol{x}; \boldsymbol{p}{i}))[y'])}$. In other words, with prompts $\boldsymbol{p}{i}$ adapting the feature space (see Fig. 3), TAP is essentially classifying the instructed representation ($\boldsymbol{h} \in \mathcal{H}^c_{i}$ in Eq. (12)), while TII * WTP is performed on uninstructed representation ($\hat{\boldsymbol{h}} \in \hat{\mathcal{H}}_{i}^{c}$ in Eq. (11)).

Authorsrebuttal2023-08-19

Author response (part II)

**Q2-3: Regarding the author's comment A2, I don't see the distinction made by having the $\theta$ condition in the probability. The probability decomposition as well as all the theoretical results in [1] remain the same with or without $\theta$.** A2-3: It is worth noting that when having $\theta$ condition in the probability of TII, WTP and TAP, prompting could change the original input space (i.e., the uninstructed representation) as the instructed representation. Since TAP classifies the instructed representation over all observed classes, it is not equivalent to TII * WTP that is performed on uninstructed representation (please refer to A2-2 above for more details). As a result, the probability decomposition in [1] (i.e., TII * WTP) will be evolved into our multi-objective decomposition max[TII * WTP, TAP], so as to improve continual prediction. A detailed proof of these results is included in Appendix A, which has been double checked to ensure solidity. We thank you again for these valuable feedback. We will add the discussions in A2-1, A2-2 and A2-3 to make our theoretical results clearer. **Q2-4: Misc. Figure 3 is presented, but it is not referenced in the paper.** A2-4: Thank you for pointing out this. Fig. 3 is an illustration of our method (HiDe-Prompt) coupled with Sec. 4.2. We will clarify it in line 241-242.

Reviewer tgkY2023-08-15

Thank you for the rebuttal

I would like to thank the authors for answering my concerns and questions. I will remain at a score of 7 and recommend this paper be accepted to NeurIPS 2023. Congratulations on the great work.

Reviewer 6q9Z2023-08-18

Thank you for your rebuttal.

I appreciate the authors’ effort to fully address my concerns. I have also read the rebuttal to other reviewers. The generality of the proposed method to other PEFT strategies (e.g., Lora) for continual learning is impressive. I believe this work could be influential in continual learning since efficient fine-tuning is very important for avoiding forgetting.

Authorsrebuttal2023-08-18

Thank you

We are happy to know that the reviewer recognizes the contribution and broad impact of our work. We also appreciate the positive feedback and strong support. Best, Authors

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC