Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings

Summary

This paper tries to address and analyze the challenge of task confusion (TC) in class-incremental learning (class-IL). This paper proposes the Infeasibility Theorem that demonstrates that achieving optimal class-IL through discriminative modeling is impossible due to TC, even if CF is prevented​. It further proposes the Feasibility Theorem, which shows that optimal class-IL can be achieved with generative modeling, provided CF is prevented. The authors further emprically assess their theorem with traditional class-IL strategies, including regularization, bias-correction, replay, and generative classifier.

Strengths

The writing of paper is clear. The paper focuses on an interesting view: task confusion and proposes rigorous theorem to analyze it. The theoretical contribution of this paper is significant. Researchers can use the theorem to guide their method desgin. The authors further assesses traditional continual learning strategies from the view of task confusion.

Weaknesses

1) I recommend that the authors clarify their theorems, experimental results, and contributions in the introduction section. Additionally, the method comparison should be detailed in the related works section. 2) The authors should give more details about discrimative/generative modeling and related continual learning strategies. Some fresh readers may not understand it.

Questions

See the weaknesses.

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

4

Limitations

I do not see any potential negative societal impact of their work.

Summary

This paper presents a novel mathematical framework for class-incremental learning and prove the Infeasibility Theorem, showing optimal class-incremental learning is impossible with discriminative modeling. While generative modeling can achieve optimal class-incremental learning with the Feasibility Theorem. The analysis suggests that adopting generative modeling is essential for optimal class-incremental learning.

Strengths

1. The motivation is strong and clear, and the importance of such theoretical framework is significant. 2. The proposed framework is insightful and well-structured.

Weaknesses

1. The proofs in appendices are informal with few equations. 2. In the appendix, Figure F.1 is missing, only the text *SIC.pdf* is presented.

Questions

1. Can you elaborate more on how the generative classifier promises optimal class-incremental learning?

Rating

6

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

The proofs should be more formal to make the theoretical framework complete.

Summary

The paper proposes a Mathematical Framework for class-incremental learning in discriminative and generative modelings, presenting a Infeasibility Theorem for discriminative models and Feasibility Theorem for generative modelings.

Strengths

The paper is easy to understand. It offers a Mathematical Framework for modeling class-IL problem.

Weaknesses

1. spelling error: Bias-Correction Impotence Corollary, "impotence" means "importance"? and in Corollary 1 (Catastrophic Forgetting) miss a inequality sign, and the same problem in corollary 2 ? 2. The Infeasibility theorem is hard to understand, the grammar seems wrong? "The CF-optimal class-IL model in Definition 3 is not be optimal loss are incompatible." 3. proof in Appendix E is unreadable, eg. “SIC.pdf”? The problem modeling of class IL is good, however, the conclusion and corresponding prove needs more explanation, which is hard to follow, especially the prove in Appendix E. The most important prove part of this paper is put into the Appendix E, while Appendix E is too simple with only oral expression.

Questions

No more questions.

Rating

5

Confidence

2

Soundness

2

Presentation

3

Contribution

2

Limitations

The limitations are proposed in the paper.

Thanks for the Response

The authors' response resolves my concerns, I decide to keep my rating.

Thanks for your feedback

We appreciate your engagement. It seems like the score is still 4. Thank you very much.