IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning

Reply to the Reviewer siLj (1/2)

We appreciate the reviewer for taking the time to provide valuable insights for our paper. We are glad that the reviewer found the paper easy to follow. Below we address the concerns raised by the reviewer to improve the confidence in our paper. > Continual learning in the context of the used baselines is a mature field with many existing works. However, the method does provide SOTA results across the board to warrant a contribution that offers a significant performance boost We are delighted to find that the reviewer recognizes the performance boost achieved by IMEX-REG across all the continual learning scenarios, especially in low buffer regimes. We further conducted several analyses of our method that emphasize on the robustness in real world scenarios (Section 5) and other characteristics such as stability-plasticity tradeoff, model calibration (Appendix C). > Theoretical justifications of the paper are not novel and mostly are reiterating previous results. Doing so is OK but does not offer any new theoretical contributions. We do not claim that the theoretical insights in our paper are novel. Rather these are provided to support our hypothesis that forms the base for our design choices. For example, Conjecture 1 provides insights that CRL and cross-entropy learning share a common hypothesis class that is essential for the proposed novel regularization strategy. This hypothesis is later validated empirically in the Experimental Results (4.2) section. We believe that providing these theoretical justifications strengthen the confidence of the readers in our hypotheses. > The connection between the proposed approach and the nervous system is very loose and emphasis on this aspect is overstated. What is the reason behind this emphasis without providing much evidence to support it? We beg to differ with the reviewer that we overstate the connection between the proposed approach and the learning in humans. However, we do draw inspiration from learning mechanisms in humans as they are the most efficient and robust continual learners out there. Learning in humans is driven by several inductive biases that encourage learning to prioritize solutions with certain properties [1]. Similarly in Deep Learning several algorithmic or architectural choices like convolutions, deep architecture bring in certain inductive biases into the learning objective [1]. We study the domain of continual learning and find that recent rehearsal based approaches suffer from generalization under low buffer regimes. We employ several implicit and explicit regularization strategies to mitigate this problem. CRL, Multi-task Learning and explicit consistency regularization provide inductive biases to learn better representations from limited buffer data to preserve previously learnt information while adapting to novel knowledge. We attribute the significant performance boost of IMEX-REG in several challenging scenarios to these inductive biases. > Q: It is important to study the effect of \alph, \beta, and \lambda on the performance. How the user should tune them? A study should be offered for this purpose. If the performance is sensitive with respect to the values of these hyperparameters, then it is essential to provide a solution for selecting the optimal values. We perform grid search to find the optimum hyperparameter and Table 6 in Appendix shows the hyperparameters used for our experiments. We agree with the reviewer that a comparative study on hyperparameter tuning would provide more insights into IMEX-Reg’s performance. Table 7 in the revised manuscript provides hyperparameter tuning experiment results. Due to limited computational capacity, we will include an exhaustive study in the final revision. In Table 7 in the revised manuscript, we can see that the model is fairly robust to the choice of hyperparameters and still outperforms the next best baseline across all the hyperparameters searched. However the model shows a higher decrease in performance as $\lambda$ value increases. Higher the $\lambda$, higher the restriction on the model to preserve old knowledge, thus limiting its ability to learn new information. In such cases, we can look at the task-wise performance of the model or stability-plasticity trade-off (Appendix C.2) to find the optimum balance between preserving learnt knowledge and adapting to novel information.

Reply to the Reviewer bb13 (1/2)

We thank the reviewer for taking time to review our paper. Please find below our response to the questions raised: > The parameters of the shared model and classifier are not defined. Section 3 and Figure 1 both describe the parameters associated with shared model and classifier. Our CL model Φθ = {f, g, g′, h} consists of a shared backbone f, a linear classifier g, an MLP classifier projection g′, and a projection head h. The classifier g represents all the classes that belong to all the tasks, and the projection head h captures the embeddings of the ℓ2-normalized representation. > Eq.1 is not clear to me. What is the actual network for f and g? Why is h not used in Eq.1? Section 4.1 provides information on particulars of f and g. We use Resnet-18 as as a backbone f and an MLP classifier as g. In Equation 1, we keep the information generic as any other backbone or classifier can be used in its place. > In the text below Eq.2, you said z = h(f(.)). However, z is not defined in Eq.2. The input and output patterns for the models f, g and h are unclear. We define z = h(f(.)) in the text below Equation 2. z represents any arbitrary 128-dimensional ℓ2-normalized projection from CRL head h. > Why introduce the existing Conjecture 1? Does this theory connect with your actual design? We introduce Conjecture 1 to provide more intuition behind our choice of auxiliary task for multi-task learning in a CL setting. Assuming that the tasks in MTL share a common hypothesis class, sharing representations across tasks primarily benefits tasks with limited training samples. Given semantic preserving augmentations, both CRL and supervised learning capture similar discriminative features. Through Conjecture 1, we hypothesize that both CRL and cross-entropy share a common hypothesis class and sharing representations across these tasks especially benefits CL under low buffer regimes. > Why the classifier can create the function spaces? Deep learning classifiers create rich function spaces by leveraging hierarchical representation, non-linear activation functions, and automatic feature learning. The layered structure allows them to capture progressively abstract features, while non-linearities enable modeling complex relationships. Automatic feature learning adapts to data intricacies, and parameter sharing fosters generalization. With end-to-end learning and substantial model capacity, deep neural networks efficiently represent intricate patterns in a data-driven manner, making them adept at handling diverse and high-dimensional datasets.

Reply to the reviewer B5E5 (1/2)

We thank the reviewer for insightful feedback and relevant references. Please find below our response to the questions raised: > The contribution of the paper is merely incremental We thank the reviewer for relevant references with respect to contrastive learning in CL. The references [1,2,3,8] use contrastive learning in their learning objective. As we have repeatedly mentioned in the paper, we do not claim any novelty emanating from contrastive learning objective. We are very much aware of the competing literature that employs Exponential Moving Average (EMA) and Contrastive Representation Learning (CRL) in CL. Specifically, CLS-ER [7] employs EMA while Co2L [2] employs CRL. Going one step ahead, OCDNet [6] is the closest prior art that employs both CRL and EMA. We respectfully disagree with the idea that imposing regularization on the contrastive projected outputs implicitly improves classification performance. Our novel contribution mainly lies in leveraging desirable traits of learning in the unit hypersphere through explicit classifier regularization in the function space. To the best of our knowledge, there are no works that leverage traits of learning in the unit hypersphere in CL. To this end, we align the geometric structures within the classifier hypersphere with those of the projection head hypersphere to compensate for weak supervision under low buffer regime. We carefully curate the list of baselines based on their learning objectives: contrastive learning methods, EMA based methods and approaches that use both contrastive learning and EMA. We note that these are highly cited works in CL that are published in reputed conferences. We compare and contrast such methods (CLS-ER, CO2L, OCDNet) in ‘Related Works’, and in experimental evaluation in Tables 1 and 2. We also compare with OCDNet in terms of task-recency bias, robustness against natural and adversarial corruptions in Section 5. Table 3 highlights the contribution of IMEX-Reg in Seq-CIFAR100 for a buffer size of 200. As can be seen from our experimental results and analysis, IMEX-Reg clearly outperforms these competing methods that entail EMA and CRL. This further emphasizes that our novel loss brings additional benefits not just in terms of performance but also in terms of robustness and less bias toward recent tasks. > The method introduces many hyper-parameters As IMEX-Reg entails multiple learning objectives, it introduces several hyperparameters. Table 6 provides the hyperparameters for used IMEX-Reg to reproduce the results reported in Table 1. We agree with the reviewer that a comparative study on hyperparameter tuning would provide more insights into IMEX-Reg’s performance. Table 7 in the revised manuscript provides hyperparameter tuning experiment results. Due to limited computational capacity, we will include an exhaustive study in the final revision. > Rehearsal-free methods such as [3, 4] already significantly outperform the proposed method The method in [3] explores possibilities of integrating the existing parameter isolation based continual learning (CL) method HAT or Sup with the strong OOD detection method CSI. As the reviewer rightly pointed out, these approaches attain accuracy higher than the existing replay-based methods. The improvement in performance can be attributed to several aspects of training: HAT and Sup entail parameter isolation during training. That inherently assumes task boundary information during training. It is a stricter constraint as task boundaries are not always accessible in real world CL scenarios. Secondly, these methods involve two stage training wherein the backbone is trained in the first stage and is kept frozen in the second stage. An additional rotation prediction and averaging is done to train a classifier in the second stage. As discussed in Table 2, Generalized Class-IL (GCIL) exposes the CL model to a more challenging and realistic learning scenario by using probabilistic distributions to sample data from the CIFAR100 dataset in each task. The aforementioned methods would fail in such scenarios due to the necessity of accessing task boundary information. As having the features lie on the unit-hypersphere leads to several desirable traits, we would like to re-emphasize that our novelty lies in leveraging activation correlations in the unit-hypersphere of the CRL to guide the classifier towards generalization. Although the above methods attain higher accuracy, they do not address the problem we have at hand. > Based on a paper under review, [5] achieves 74+% in Seq-CIFAR10 with a buffer size of 200 in ResNet-18. The authors may compare their method with [5]. We thank the reviewer for suggestions on the related work. We will include this in the final revision.