Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed to alleviate CF. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experience rehearsal, and context-dependent gating, rarely resulting in CF. Inspired by how the brain exploits multiple mechanisms concurrently, we propose TriRE, a novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered in isolation.
Paper
Similar papers
Peer review
Summary
The paper concerns catastrophic forgetting in neural networks and draws from neural mechanisms contributing to the absence of such phenomena in the brain to propose a method to overcome CF through targeted neuron retraining, task knowledge revision, and enhanced learning for less active neurons. The authors present an impressive suite of experimental results, benchmarking their proposed method as superior or comparable to notable existing approaches in class- and task-IL scenarios and with further analysis of the different stages of the method.
Strengths
+ The work creatively and effectively links to biological motivations toward continual learning (CL) capabilities. + Fairly inclusive summary of existing CL approaches with contributive discussion as to their strengths and weaknesses toward integration in the proposed approach. + Figures are well-crafted and easily understandable. + Performance is comparable to SOTA methods. + It's nice to see in the experimental data the significance of dynamic masking, retaining, rewinding.
Weaknesses
- (As mentioned by the authors:) The work is primarily applicable only to CNN-based architectures in its present state, and additional tuning is required for the increased number of hyperparameters. While the paper is nice, one can see how TriRE won't work on transformers due to computational expense. - Regarding Related Work, the authors could elaborate on whether there are any other notable CL approaches that draw upon strengths of previous works as TriRE does and clarify how TriRE is superior/comparable (if applicable). - The authors could perhaps move discussion of limitations more heavily to the main text as opposed to the Appendix. The limitations discussed in Appendix C pertain to the practical applicability of the work and thus have merit for inclusion.
Questions
* I am curious as to why rewinding improves accuracy, perhaps this could be another direction for exploration. * Small clarifications regarding the method itself: * Learn: finite replay buffer with loss-aware experience rehearsal? * Retain: parameter isolation where important connections and weights are learned? * Revise: How are the subnetworks combined? I wonder if there is a way to specifically focus on free neurons when learning new tasks, so during the combination stage they are orthogonal to the combined network? * Rewind: Does relearning target specific unactivated neurons/parameters? This step is perhaps redundant if simply taking weights from a few epochs before and training more on new mini-batches to relearn. Minor: * Lines 159-160: Put “k” in math mode (top-k, k-winner) * Line 215: “Retatin” -> “Retain”
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
3 good
Limitations
The authors touch upon the limitations of their work in the main text and more so in the Appendix. Social impact — not directly applicable but is mentioned in broader impacts discussion.
Summary
The paper proposed a new continual learning (CL) method that updates partial neurons while rewinding other neurons to the previously stored weights. To do this, the authors utilize sparsity constraints to the network weights and select highly activated neurons. Consequently, the proposed method outperforms the baselines.
Strengths
1. The proposed method is generally technically sound. 2. The ablation study in Table 2 helps us understand the contribution of each component.
Weaknesses
1. The proposed method has three components, retain, revise, and rewind. The key contribution part is rewind while other parts are periphery or have no novelty. 2. What is the overhead cost for rewinding, such as training time, memory cost for holding previous weights, and the performance drop for the current task? 3. There are 7 different parameters and it seems that it is arbitrarily chosen to make the best performance for each task (Appendix D) as the authors mentioned in Appendix C, which is appreciated. 4. The statement in L211-212, “This is helpful because studies show that in the human brain, less active neurons follow a ‘use-it-or-lose-it’ philosophy” is difficult to understand why it is “helpful.” It seems that the authors blindly think that “human-like” is good. 5. It will be better if the authors include recent baselines [1-2] and check whether the proposed method is state-of-the-art in the current setting. [1] Fu-Yun, et al. "Foster: Feature boosting and compression for class-incremental learning." ECCV. 2022. [2] Zhou, Da-Wei, et al. "A model or 603 exemplars: Towards memory-efficient class-incremental learning." ICLR. 2023
Questions
See Weakness
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
2 fair
Presentation
3 good
Contribution
2 fair
Limitations
See Weakness
Summary
The paper proposes a method for avoiding catastrophic forgetting in continual supervised learning that operates in three stages. In the first stage (retain), a subnetwork for the current task is identified by detecting most/least activated neurons of the main network. In the second stage (revise), both the main network and the extracted subnetwork are re-trained / fine-tuned with examples from the current task and examples from a replay memory. The subnetwork is later integrated into the main network. In the final stage (rewind), the weights belonging to non-cumulative subnetworks are finetuned for a few epochs. The paper presents experimental results on three benchmark continual learning datasets, in both the task-incremental and class-incremental settings, by also comparing with counterpart methods in the area. --Rebuttal-- I read the rebuttal, along with other reviewers' comments, and increased my score accordingly (to borderline accept) during the rebuttal phase.
Strengths
- The originality of the paper relies on a new multi-stage method to tackle catastrophic forgetting which relies on the important concepts of modularity and example replay. - The paper is in general easy to follow. - The paper is properly aligned with literature in the area.
Weaknesses
- Although the proposed approach touches on a lot of important points in catastrophic forgetting and continual learning, it seems to be a mere combination of existing ideas into a so-called three-stage approach, therefore undermining the novelty of the proposed mechanism. Furthermore, what seem to be novel aspects of the proposed approach are not explained in full detail, and therefore it is difficult to estimate their impact. For instance, in the retain phase, activation pruning is performed by using the existing heterogeneous dropout, while weight pruning used the existing CWI approach. For activation pruning, in lines 158-159, it is stated that a counter is used to determine the top k-winner activations. However, not technical details are provided regarding how this counter works. In the revise stage, lines 191-193 mention that the learning rate is "considerably" reduced at this phase; however, no insights onto how to decide what the reduction rate to use. For this same stage, in lines 195-199 it is mentioned that the S and S_{t} subnetworks are eventually merged; however, no technical details of how this merge occurs are provided. Finally for the rewind stage, in lines 209-214, it is unclear how to decide for how many epochs (k?) to rewind the network, what are the criteria, the requirements to decide this. - The experiments lack analysis of computational cost (e.g. memory consumption, training time). As can be inferred from Algorithm 1 and section 3, the multi-stage procedure proposed in this paper involves several calculations and passes through the network/subnetwork. What is the cost of this multi-stage procedure? How does this compare to counterpart methods? - In the experiments, some results are left unexplained while some others have an ambiguous explanation. For example, why does the proposed method seem to underperform on Task-IL for two of the three datasets evaluated? The explanation for the "miserably" (please change this word) performance of methods such as LwF and SI provided in lines 240-246 looks ambiguous and incorrect, since these methods have been used previously in Task-IL. - From the results in Figure 3, it can be implied that the proposed method actually improves performance as the number of tasks increases. Is that the case? Did you run multiple task orders? Are tasks 10 to 20 naturally easier?
Questions
Please refer to questions listed in the "weaknesses" section.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
2 fair
Contribution
2 fair
Limitations
Limitations in terms of the network architecture used are clearly stated in the paper.
Authors' rebuttal
Thanks to the authors for their responses to my questions and concerns. After reading those, along with other reviewers' comments and responses to those comments from the authors, I am happy to increase my score as long as the authors commit to include in their final version of their paper the explanations and results of missing elements, in particular computational cost of the method compared to others and details of hyperparameter tuning (considering the large number of hyperparameters that are needed).
Reply to reviewer Soxi
We thank the reviewer for their response. As suggested by you and other reviewers, we fully commit to including experiments on computational overhead, hyperparameter tuning, and explanation for missing elements. As can be seen in official comments for Reviewer 6DDf, we have made some progress with respect to suggested changes. Please find our complete revision plan under an official comment for Reviewer tURo. Please let us know in case any of your concerns are missing in the final revision plan.
Summary
This article introduces a multi-faceted approach to continual learning. combining features of many other approaches and relying on a three-phase training process, such that, as each new task in a sequence of tasks is encountered, subsets of weights in the network are 'retained', 'revised' or 'rewound' to values that maintain old knowledge while beginning to learn the new task, facilitate integration of weights important for the new task with those important for previous tasks, and maintain plasticity for future learning, respectively. The authors find advantages for this approach relative to a large range of other approaches, especially with more complex task settings, and perform several ablations helping to clarify the roles of the three stages, along with a few other explorations.
Strengths
The paper cites a wide range of relevant related work and situates its approach within the context established by other approaches. The other approaches considered seem fairly extensively sampled. I appreciated the consideration of the effects of the different ablations which show the importance of the 'rewind' phase as well as the stability plasticity analysis shown in the appendix.
Weaknesses
The work seems thoughtful in relation to the relevant existing literature but it may be that the CL paradigm as explored in this project is in need of re-framing, if new breakthroughs are to be achieved. While the advantages relative to many of the baselines seem clear, I felt that the additional complexity of the TriRE scheme made the gaps between it and some of the other approaches relatively difficult to get excited about. Some of the features that seem relatively important, such as CWI, are direct importations from previous work, making it difficult to assess whether TriRE really advances our understanding, given its additional complexity compared to most other work. To me the greatest weakness of the approach is one that appears to be widely shared across the CL literature: This is the fact that this and all of the cited work rely on triggering complex meta-processes at the boundaries between tasks in ways that seem very distant from what might occur in biological networks on in naturalistic continual learning settings where task boundaries are not announced. We need approaches that can address the continual learning problem when tasks shift more gradually and task boundaries are not available. The approach also exploits features limited to the setting in which tasks are defined by the fact that they use completely non-overlapping sets of output neurons (i.e. distinct class labels). For example the Loss on the new task items in Eq 4 only considers the class labels relevant to the current task. Finally, continual learning as explored in this literature is restricted to tasks with no intrinsically cumulative structure. Catastrophic forgetting also occurs in settings where new learning could productively build on structure learned in previously learned tasks (in the way that addition builds on counting, multiplication builds on addition, etc). The whole body of work thus seems narrowly focused on paradigms of limited general interest.
Questions
I had difficulty understanding the CLS-ER results, which are almost as good as TriRE. A clearer understanding of what the CLS-ER model shares with TriRE and how it differs from it would be useful in evaluating the contributions of TriRE over and about the use of an EMA model smoothing weights across tasks. The note in Table 1 is insufficient for me to understand the relationship between CLS-ER and TriRE.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
I initially wrote: "The paper mentions limitations in the appendix. As stated they hint at some of the same limitations I see in the work as described above. Per the importance placed on limitations in instructions to authors and reviewers, these ought to be placed in the main text." The authors have addressed this concern through the discussion during the rebuttal, leading me to increase my rating to 'Accept'. While I think the weaknesses described above still apply, they are, as I have said, largely shared by a mini-paradigm in which these issues have been addressed. I hope the authors will be encouraged by acceptance of this work to work on a new paradigm.
What revisions do the authors plan?
I hope I'm not missing it but I didn't see signs of any intention to revise either in the overall rebuttal or in the response to my comments. Before re-confirming my belief that the paper deserves consideration for acceptance, I'd like to see the authors present a revised ms with a limitations section included in the main paper, specifically addressing the concerns I have raised with the whole continual learning setup of using announced task boundaries to gate complex meta-processes.
Reply to Reviewer tURo
We thank the reviewer for swift response. We take reviewers’ feedback seriously and intend to accommodate all their concerns in the final revision. As uploading a revised manuscript is not a possibility due to NeurIPS guidelines and the rebuttal had a limited word count, we did not provide a revised version of the limitations section. Considering the feedback from all reviewers, we intend to update our manuscript with the following changes: - Computational and memory overhead comparison - Robustness to choice of hyperparameters - Information pertaining to differneces between CLS-ER and TriRE - Clarifications regarding Figure 3 - Extending our evaluation with more recent baselines (as proposed by other reviewers) - Other miscellaneous / minor changes, clarifications - Limitations and Future Work: We proposed TriRE, a novel CL paradigm that leverages multiple orthogonal CL approaches to effectively reduce catastrophic forgetting in CL. As orthogonal CL approaches may not always be complementary, the selection of such approaches needs careful consideration in TriRE. In addition, having multiple objective functions naturally expands the number of hyperparameters, thereby requiring extensive tuning to achieve optimal performance. Therefore, additional computational complexity and memory overhead due to the staged approach and extensive hyperparameter tuning are one of the major limitations of the proposed method. For the same reason, we highlight that TriRE is not directed towards compute-intensive architectures such as vision transformers. As TriRE involves different stages of training within each task, it assumes the knowledge of task boundaries. Moreover, in line with state-of-the-art methods in CL, each task entails a non-overlapping set of classes with data within each task shuffled to guarantee i.i.d. data. However, in the case of online learning where data is streaming and the distribution is shifting gradually, TriRE cannot be applied in its current form. Therefore, additional measures such as task-boundary approximation and modification to learning objectives are necessary to enable TriRE to work in such scenarios. Furthermore, traditional CL datasets considered in this work entail independent tasks and data points without intrinsic cumulative structure. As TriRE does not leverage structures learned in previously encountered tasks, structure learning forms one other limitation of this proposed method. Reducing computational and memory overhead, extending to task-free CL scenarios with recurring classes, and leveraging intrinsic structures within underlying data are some of the future research directions for this work.
Thank the authors that respond to my concerns. I have follow-up questions. `Related to Overhead of rewind` I recommend measuring or computing the complexity of the overhead cost in terms of memory and runtime. `Related to `It will be better if the authors include recent baselines [1-2]` It would be better if the authors specify how the experimental setting is different which hinders the fair comparison. Both are not quite different settings from DER++ used in Table 1
Reply to Reviewer 6DDf
We thank the reviewer for taking the time to review our rebuttal. With regard to measuring computational and memory overhead, we plan to include this in the final revision (See our 'revision plan' in the official comment for Reviewer tURo). As these experiments need to be standardized (with experimental settings, software environments, and underlying hardware) to make sure we are comparing apples to apples, we expect these experiments to take some time. We hope to provide these results before the discussion period ends, but unfortunately, time and limited computational capacity are proving to be a constraint. In any case, we will include these results in our final revision. We considered the most important baselines from different CL approaches to evaluate the efficacy of TriRE. We intend to compare and contrast Foster [1] and Memo [2] analytically in our final revision. Some of the key differences hindering our progress are (1) Different backbones (ResNet-18 in TriRE Vs ResNet-32 variant in Foster for CIFAR100 experiments), different buffer sampling strategies, different buffer sizes (500 Vs 2000) and different training schedules. The same arguments hold for Memo as well as it has similar experimental settings as Foster. With sufficient time, these differences can be addressed and methods can be compared on a common ground. As per the reviewer's suggestion, we will include an experimental evaluation entailing a comparison between these methods in the final revision. Our revision plan can be found in the official comment for Reviewer tURo. [1] Fu-Yun, et al. "Foster: Feature boosting and compression for class-incremental learning." ECCV. 2022. [2] Zhou, Da-Wei, et al. "A model or 603 exemplars: Towards memory-efficient class-incremental learning." ICLR. 2023
Requesting feedback from Reviewer 6DDf
As per your suggestions, we are working on experiments concerning computational and memory overhead, and the comparison between Foster and Memo. As the discussion period is drawing close, we kindly request your feedback. Please let us know if any concerns remain. We would be more than happy to provide additional information to ensure complete comprehension of our work. If otherwise, we kindly request that you adjust your score to reflect the improved confidence in our paper.
Comparison with Foster [1] and Memo [2] as requested by Reviewer 6DDf
Based on reviewer's suggestions, we provide a comparison between Foster [1], Memo [2] and TriRE in Class-IL. The experiments are conducted on Seq-CIFAR100 with buffer size 200 with 5 tasks and 3 random seeds. We caution that the implementation details are slightly different for these methods and defer the details to the the final revision. | Foster | Memo | TriRE | | ------------- | ------------- | ------------- | | 40.48 ± 0.53 | 43.57 ± 0.44 | __43.91 ± 0.18__ | Foster entails a dynamic expansion and compression to accommodate new information. Similarly, Memo entails expansion in the later layers while preserving generic information in the earlier layers. On the other hand, TriRE promotes generalization through a combination of weight and function space regularization, selective forgetting, and relearning thereby producing superior performance across tasks. As can be seen, TriRE outperforms both Foster and Memo. As the CL training progresses, both Foster and Memo are constrained to accommodate new information in limited number of parameters resulting in lower performance in later tasks. However, TriRE suffers from no such limitation resulting in superior performance. The task wise performance can be found in the following anonymous link: [taskwise_performance](https://osf.io/pyfxw/?view_only=db15d0cbc14644febfa43068f4fa992a) We will update the above results with a detailed discussion and the figure in our final revision. [1] Fu-Yun, et al. "Foster: Feature boosting and compression for class-incremental learning." ECCV. 2022. [2] Zhou, Da-Wei, et al. "A model or 603 exemplars: Towards memory-efficient class-incremental learning." ICLR. 2023
Regarding computational and memory overhead as requested by 6DDf
We conduct a comparative analysis of the learnable parameters and memory required by TriRE in contrast to those of DER++, EWC and PNNs, (i.e each individual family of CL methods). Firstly, similar to DER++ and EWC, TRiRE does not add any learnable parameters to the model. However, it is evident that PNNs have an infeasible amount of learnable parameters which gets progressively worse with longer task sequences. Secondly, the observed increase in memory consumption in TriRE can be attributed to several factors: (1) the application of multiple masking mechanism for parameter isolation, (2) the incorporation of the Rewind phase necessitating weight retention from a previous epoch, and (3) the utilization of the Exponential Moving Average (EMA) model to enhance knowledge consolidation. All of these factors hold memory but does not add any learnable parameter to the training. It is also noteworthy that, the employed masking mechanism presents potential for further optimization. Future iterations could explore techniques like spatial hashing and compression to enhance the computational efficiency of TriRE. Additionally, TriRE exhibits scalability for longer task sequences, distinguishing it from PNNs. As illustrated in the accompanying table, the memory consumption in PNNs experiences exponential growth with the number of tasks, whereas TriRE maintains a fixed capacity model. This is because our work, through the Revise and Rewind phases, compels the model to learn the joint task distribution while also facilitating the involvement of less active neurons in the learning process throughout training. | Methods | | Learnable Parameters (Million) | | | Memory Consumption (Million) | | |---|---|:---:|---|---|:---:|---| | | **5 Tasks** | **10 Tasks** | **20 Tasks** | **5 Tasks** | **10 Tasks** | **20 Tasks** | | DER ++ | 1x | 1x | 1x | 1x | 1x | 1x | | EWC | 1x | 1x | 1x | 3x | 3x | 3x | | TriRE | 1x | 1x | 1x | 6x | 6x | 6x | | PNNs | 27x | 79x | 240x | 27x | 79x | 240x | In summation, TriRE occupies an intermediary position among the aforementioned methodologies with regards to the learnables parameters and memory overhead. TriRE is as good as DER++ and EWC in terms of learnable parameters required but does significantly better than PNNs. It offers superior performance with increased memory overhead over DER++ and EWC, all the while establishing a novel precedent for the harmonious integration of various CL techniques and without capacity saturation as in PNNs. We will include this analysis in the final version of the paper. Please let us know in case we have missed something.
Reply to Reviewer 6DDf
As the discussion period is nearing its end soon, we eagerly await the reviewer's feedback on our revised plan. We've made significant strides in our revisions: for instance, we conducted comparisons with Foster and Memo as the reviewer recommended. Additionally, we incorporated a memory overhead analysis to offer readers a comprehensive perspective on our method. Currently, we are delving into other experimental areas such as computational overhead and robustness concerning hyperparameter selection, among others. Given these enhancements, we would appreciate it if the reviewer could indicate any lingering concerns. We have made every effort to address your concerns and sincerely hope for your continued engagement. If our revisions align with your expectations, we respectfully request a reconsideration of the score.
Robustness to hyperparameters should be experimentally presented not verbally described.
The authors should show experimental results not verbally described. Otherwise, readers are difficult to believe that the proposed method is robust to hyperparameters although it requires many of them, like Figure 5.
Reply to Reviewer 6DDf
We thank the reviewer for their swift response. We appreciate the reviewer's feedback and acknowledge the importance of analysis depicting robustness to choice of hyperparameters. As mentioned earlier, Figure 5 is one such example that depicts the relationship between the choice of rewind percentile and TriRE performance. As per reviewer's suggestion, we will collate more such results with respect to other hyperparameters as well. Although we envisage to provide them before the end of the discussion period, we are constrained by time and limited computational capacity. In any case, we will report these results in the final revision.
More hyperparameter tuning experiments
In accordance with the recommendations of the reviewers, we have carried out additional hyperparameter tuning experiments and subsequently compared the performance of TriRE in the context of Seq-CIFAR100 with a buffer size of 200 and 5 tasks. The outcomes are detailed as graphs in the temporarily accessible anonymous link: [Tuning experiments link](https://osf.io/pyfxw/?view_only=db15d0cbc14644febfa43068f4fa992a) Our experimentation involved the manipulation of parameters such as sparsity, k-winner sparsity, and EMA model update frequency. As can be seen from the graphs, the performance does not vary significantly for a range of values for these hyperparameters. This observation suggests that achieving satisfactory performance is achievable without the necessity for exhaustive hyperparameter tuning. We will include these results and a description in the final revision.
Thanks
Thank you for the detailed response. Regarding making space for limitations: I am not sure what would be feasible to remove while preserving the intended message and impact of each section -- however, the initial descriptions of each of the three stages (Section 3) could be made more concise to create more space overall; the Broader Impacts section could be shortened (last 2 sentences) and/or combined with the conclusion; and perhaps Algorithm 1 could be moved to the appendix, although I am not sure this is in line with best practice.
Reply to Reviewer 3Tuq
Thank you for your swift response. In line with the reviwer's suggestion, we will be moving the Limitations section to the main paper. Furthermore, we have expanded the scope of limitations to encompass the potential challenges that could arise from a naive extrapolation of TriRE to transformer-based architectures, notably with regards to computational complexity and memory overhead (Refer our 'revision plan' in the official comment for Reviewer tURo).
Given author responsiveness re limitations and other points, I now more strongly favor acceptance
If there is an official way to revise my rating of the ms, I hope someone will let me know. Otherwise, I'll just say here that I'm hopeful the paper can be accepted and presented at least as a poster at the conference.
Thank you for advocating for our paper. In your official review session, there is an "Edit" button that will enable you to modify the initial review and rating. Your support is greatly appreciated.
Done
I have edited my initial review, noted the author's plan to address limitations in the 'limitations' section of the review, and I have revised my official rating upward with the rationale explained in the current test in the 'limitations' box.
Decision
Accept (poster)