A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm

Summary

This paper proposes a framework for domain incremental learning that unifies ideas from previous methods in this area. A theoretical analysis is first performed to formally state the problem of domain incremental learning. The theoretical formalization identifies three ways to bound the risk of error on a particular domain: 1) by a naive ERM-based bound, 2) by an intra-domain model-based bound, 3) by a cross-domain model-based bound. Based on the identified limitations of these three bound, the authors proposed a unified view of the generalization bound for domains observed in incremental learning. They propose to integrate these three bounds using coefficients that vary depending on the domains at hand. Existing methods in domain incremental learning are then categorized according to the components of the unified framework. In the second part of the paper, the authors propose a method where the coefficients are both learnable and dynamic. Experimental results on both toy data and benchmark datasets, along with comparison to SOTA methods in the area, are provided. ---Rebuttal--- I read the authors rebuttal. My review was already positive, so I left my score unchanged.

Strengths

- Regarding originality, the paper presents originality both at the theoretical and experimental levels. Regarding theoretical originality, to the best of my knowledge the paper presents the first comprehensive analysis of generalization bounds in domain incremental learning while also categorizing SOTA methods in the area according to different views of these bounds. Regarding experimental originality, the paper presents a method that is dynamic and adaptable to relevant aspects such as the memory size and the domain relatedness in domain incremental learning. - Regarding significance, the paper provides new insights on domain incremental learning through rigorous and extensive theoretical analysis. This is of high relevance for researchers in this subfield. - Regarding quality and clarity, the paper is very well written and it is easy to follow. Extensive details are provided both in the paper and in the supplementary material.

Weaknesses

- From the experimental results presented in Tables 2 and 3, the proposed method seems to be competitive but not significantly better than counterparts (especially DER++). - Details on the computational complexity/cost of the method are not provided in the formulation of the method or in the experiments. Considering that the method needs to learn the coefficients, and therefore the problem in Eq. 13 is more difficult than the optimization problem in counterpart methods, what would be the extra-cost in terms of computation?

Questions

Please refer to the "weaknesses" section for questions.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Limitations are properly addressed thorough the paper. There are not potential negative societal impacts of this work that need to be stated explicitly.

Summary

In this paper, the author proposed a unified theoretical framework in order to reveal how the tasks in the domain incremental learning are related. Moreover, the framework unifies various existing methods from the perspective of domain generalization bound and it showed that the unified generalization bound can be organized in the format of the combination of data, memory bank, and history model. Meanwhile, the author showed that the fixed coefficient of the above combination is always less effective compared to the adaptive ones. As a result, the author proposed an adaptive adjusting coefficients method to tightly bound the generalization upper bound. Extensive experiments also showed the effectiveness of the unified framework.

Strengths

1. This paper is well-organized and the mathematical proofs are sound. \\ 2. The author combined theoretical analysis of domain adaptation to deal with the domain incremental problem, which is impressive. \\ 3. The proposed unified framework can explain a variety of previous works, which is very powerful. \\

Weaknesses

see questions

Questions

1. The author should emphasize what the ghost example in supplementary material is ( if it is some kind of subset of the previous domain? or some stored replay examples ), otherwise the transition from Equation 19 to 20 is not smooth, e.g. it provided a loose scale-up. \\ 2. From Theorem 3.4, we can find that the first bounding term is actually a sum up of loss from the first model to the current one (ERM), so is that appropriate to store the model at every training stage in the sample replay setting, and is the history model froze during the training? Meanwhile, will that cause a lot of additional memory burden? \\ 3. The intra-domain and cross-domain constraint is actually an every step alignment with the history learned models, thus as long as the domain discrepancy is relatively large from one to another, the $h$ would have a high probability of severe change, making the second term of Equation 8 in the left large, would that be a problem since it seems no explicit parameter regularization? 4. Also, since in Equation 8, every step of updates would come from a bottom-up calculation from the first domain to the current one, it seems the calculation would be expensive as the growth of the domain, so I have concerns over the calculation burden. Also, as a consequence, would the method can be implemented in some large-scale real-world datasets which used in domain incremental learning, e.g. DomainNet? 5. The Theorem 3.4 seems like a combination loss over all domains, thus, I have a little concern over the novelty of this Theorem. ( Considering it is the main Theorem).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

yes, the authors adequately addressed the limitations yet

Summary

This submission investigates domain incremental learning problem, where the model is adapted to a sequence of new tasks which have different distributions. Starting from some (domain related) generalization bound theory, the combination of three different bounds could be used to address domain incremental learning, and meanwhile several existing methods could be unified in the combined bound. An objective to find the efficient combination weight is also introduced. Experiments on both toy dataset and real dataset show the efficacy of the method.

Strengths

- The motivation is clear, it is reasonable to combine the three different bounds to deal with mode general case. - It is good to derive a decent method from theoretic perspective for domain incremental learning task. And it is creditable that some hyperparameters such as $C$ could be also estimated inside the bound. - It is admirable that the resulting bound could include several existing methods, which connects the intuitive methods and the corresponding theory.

Weaknesses

Overall the submission is a good paper, and I have no obvious concerns about the submission, while I simply hope maybe the authors could add more narrations in the main paper, on the related works of domain incremental learning, as there are relatively fewer existing works in this topic. And I am also interested in the effectiveness on the huge dataset, such as DomainNet which has relatively larger domain gap and more categories.

Questions

As the generalization bounds considered in the submission are actually related to domain adaptive task (from [3]), I am curious about whether the method could be extended to the more practical scenario, unsupervised domain incremental learning, where the new domain only contains unlabeled samples. I think there is possibility to extend the work, as some unsupervised domain adaptation methods are directly utilizing the $d_{\mathcal{H}\Delta\mathcal{H}}$, such as MCD[a], though in the current setting there are only limited source samples. *reference* [a] Saito K, Watanabe K, Ushiku Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 3723-3732.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

None potential negative societal impact.

Rebuttal from authors

Thanks to the authors for their responses to my questions. I consider this paper quite a strong contribution, and one of the best readings I've had as a NeurIPS reviewer in the past 5 years. I am very supportive of this paper being accepted.