Constantly discovering novel concepts is crucial in evolving environments. This paper explores the underexplored task of Continual Generalized Category Discovery (C-GCD), which aims to incrementally discover new classes from unlabeled data while maintaining the ability to recognize previously learned classes. Although several settings are proposed to study the C-GCD task, they have limitations that do not reflect real-world scenarios. We thus study a more practical C-GCD setting, which includes more new classes to be discovered over a longer period, without storing samples of past classes. In C-GCD, the model is initially trained on labeled data of known classes, followed by multiple incremental stages where the model is fed with unlabeled data containing both old and new classes. The core challenge involves two conflicting objectives: discover new classes and prevent forgetting old ones. We delve into the conflicts and identify that models are susceptible to prediction bias and hardness bias. To address these issues, we introduce a debiased learning framework, namely Happy, characterized by Hardness-aware prototype sampling and soft entropy regularization. For the prediction bias, we first introduce clustering-guided initialization to provide robust features. In addition, we propose soft entropy regularization to assign appropriate probabilities to new classes, which can significantly enhance the clustering performance of new classes. For the harness bias, we present the hardness-aware prototype sampling, which can effectively reduce the forgetting issue for previously seen classes, especially for difficult classes. Experimental results demonstrate our method proficiently manages the conflicts of C-GCD and achieves remarkable performance across various datasets, e.g., 7.5% overall gains on ImageNet-100. Our code is publicly available at https://github.com/mashijie1028/Happy-CGCD.
Paper
Similar papers
Peer review
Summary
The paper presents a novel approach to the task of Continual Generalized Category Discovery (C-GCD). The proposed framework, named Happy, aims to address the challenges of continuously discovering new classes from unlabeled data while preventing the forgetting of previously learned classes. The authors identify two primary issues in C-GCD: prediction bias and hardness bias. To mitigate these, they introduce clustering-guided initialization, soft entropy regularization, and hardness-aware prototype sampling. Experimental results show that the Happy framework significantly improves performance across various datasets, including notable gains on ImageNet-100.
Strengths
Clear Contributions: The paper clearly outlines its contributions, making it easier for readers to understand the novelty and significance of the work. Experimental Validation: The framework is validated through extensive experiments on multiple datasets, demonstrating significant improvements over state-of-the-art methods. The 7.5% gain on ImageNet-100 is particularly impressive. Addressing Biases: The identification and mitigation of prediction and hardness biases are well-articulated and experimentally supported. This shows a deep understanding of the underlying issues in continual learning.
Weaknesses
Complexity of the Framework: While the combination of multiple techniques (clustering-guided initialization, soft entropy regularization, and hardness-aware prototype sampling) is innovative, it also adds complexity to the framework. This could make it challenging to implement and tune in practice. Scalability Concerns: The paper does not extensively discuss the scalability of the proposed framework. As the number of classes and stages increases, the computational and memory requirements could become prohibitive. Evaluation Metrics: The paper primarily focuses on accuracy improvements but does not delve deeply into other potential evaluation metrics, such as computational efficiency, memory usage, or robustness to different types of data distributions.
Questions
Generalizability to Other Domains: While the framework is validated on multiple vision datasets, it is unclear how well it would generalize to other domains, such as natural language processing or other types of sequential data. Adapt to the SoTA methods: Is it possible to adopt the method to several significant recent works in the field, such as SPTNet, TIDA, and InfoSieve?
Rating
4
Confidence
3
Soundness
3
Presentation
2
Contribution
2
Limitations
NA
Response to Reviewer cVL6
Dear Reviewer cVL6, Thanks for your feedback. Considering the "weaknesses of the other Reviewers", we have given responses point by point in the rebuttal. Here, we summarize our responses to the weaknesses pointed out by all reviewers: * **Hyper-parameters tuning.** Please refer to **General Response to Common Concern 1** and **response to you in W1**. Our method requires three hyper-parameters, and **we only need to set all weights to 1**, which is generalizable and could perform well across all datasets. Therefore, the parameter tuning is simple and easy to implement. * **Complexity of the method.** * Firstly, we have demonstrated the necessity of each module, as shown in Table 5 of the main manuscript. * Secondly, we have provided the computational overhead for our proposed modules (see **responses to you in W2 and W3**). The cluster-guided initialization, entropy regularization, and hardness modeling that we propose consume less than 1% of time and memory resources, which means that they are very efficient and effective. * **Scalability of the method.** All continual learning methods experience a linear increase in computational demand with the number of classes and stages. Compared to previous replay-based methods, our approach saves disk storage by a factor of 1/1000, as detailed in the **response to you in W2**. * **Generalization abilities of the method.** * We have conducted evaluations on 15 corrupted and shifted distributions of CIFAR100-C, as in the **General Response to Common Concern 1** and **Rebuttal PDF**. * Furthermore, previously, we have also conducted experiments on two more fine-grained datasets: **Stanford Cars** and **FGVC Aircraft** datasets, with the results as follows: | Standford Cars | All | Old | New | | :------------: | :-------: | :-------: | :-------: | | GCD | 47.00 | 47.73 | 42.61 | | MetaGCD | 54.67 | 55.28 | 50.95 | | Happy (Ours) | **62.79** | **63.68** | **57.34** | | FGVC Aircraft | All | Old | New | | :-----------: | :-------: | :-------: | :-------: | | GCD | 42.95 | 44.35 | 33.38 | | MetaGCD | 47.16 | 48.61 | 38.23 | | Happy (Ours) | **53.10** | **53.81** | **48.71** | * To conclude, our method achieves remarkable performance compared to the previous sota on 6 datasets and 15 unseen distributions, showcasing the strong generalization abilities of Happy. * **Adapting to SoTA methods.** We have adapted our method to two sota methods: SPTNet and InfoSieve. Please refer to **response to you in Q2** for more details. We will add all results of adapting Happy to all three methods you listed in the final version. **We will also add citations of the three methods: SPTNet TIDA, and InfoSieve in our reference list.** Thank you again for your feedback. We hope our responses and extensive experiments have addressed your concerns. If you have any further questions, we welcome continued discussion. Thank you once again for your review.
Dear Reviewer cVL6, Thanks for your feedback. As Reviewer TLvc and jUDg mentioned, our responses have addressed their concerns. We were wondering if our latest responses have addressed your concerns. If you have further specific questions on the weaknesses of the method, feel free to raise them and we are willing to discuss them with you. We look forward to and appreciate your feedback. Thank you once again.
Thanks for the authors' response. The lack of a solid theoretical foundation and not quite stable debias mechanism affect model reliability. I'm also not quite sure how much hyper-parameters tuning would further affect its stability. Even though they're all ones, but these may coincidentally be optimal.
Further response to Reviewer cVL6
Dear Reviewer cVL6, Thank you for your feedback. We will respond to your concerns point by point. * Lack of a solid theoretical foundation. * Actually **we have included the theoretical foundation of our method**. Please refer to **Further Response to Reviewer TDQJ**, we also include this point as below. * The theoretical foundation of our method is the **InfoMax principle** [R1], namely **maximizing the mutual information (MI) between the inputs and outputs of a system.** Here inputs and outputs denotes the feature $Z$ and model predictions $Y$. * For the self-training problem of learning new classes in C-GCD, we employ the maximizing the mutual information (MI) theory. Specifically, the objective can be expressed as $\max I(Y, Z) = H(Y) - H(Y|Z)$, where $Y$ represents the labels and $Z$ the features, with $I$ and $H$ denoting mutual information and entropy, respectively. * The proposed entropy regularization on model predictions $Y$ is the minus of first term $-H(Y)$, while the self-distillation objective aims to encourage model to predict more confidently, resulting in lower prediction entropy, namely the second term of minimizing $H(Y|Z)$, namely maximizing $-H(Y|Z)$. * Overall, our method follows the Infomax principle and aims to maximize mutual information, to ensure the quality of self-learning. * Not quite stable debias mechanism affect model reliability. * Instability is an inherent issue in C-GCD; however, our approach has effectively mitigated this instability, exhibiting significantly improved stability compared to previous methods. * **Example 1:** On CIFAR100, our method improves the hardest accuracy from 65.10% to 70.23%, and the variance decreases from 23.04% to 10.33%, which means our method greatly mitigates the forgetting of hard old classes and alleviates the extent of imbalance caused by hardness bias. * **Example 2:** Our method exhibits significantly less variability compared to others,On CIFAR100, the maximum new accuracy difference across stages is $\Delta_{new}$=6.3 (56.1-49.8), which is less than the 17.3 (48.9-31.6) of **previous sota** MetaGCD. Similar patterns can be observed in other datasets as well. * Our method is robust and generalizes well to 15 unseen distributions, please see **Rebuttal PDF**. * Overall, our method is a relatively robust method which is much more stable than previous methods. * Hyper-parameter tuning. * As we responded in **General Response to Common Concern 1** and **Further Response to Reviewer TLvc**, we did not claim that the hyper-parameters are all optimal at 1, but rather, they are **near-optimal**, which is sufficient to achieve SOTA results across 6 datasets and 15 unseen distributions, without the need for meticulous tuning, thus saving on computational costs. * For $L_{new}$ and $L_{old}$, our results show that a 1:1 weight ratio is optimal, because both old and new classes matter and we need to maintain a balanced focus on both new and old classes as a whole (other weight ratios of $L_{new}$ and $L_{old}$ will result in a 5% to 10% performance degradation). So keep the weight ratio of $L_{new}$ and $L_{old}$ to 1, and the only parameter for tuning is $\lambda$ for $L_{con}^u$. We found that $\lambda$ equals 0.7$\sim$1 works fine. Therefore, It is simple to choose hyper-parameters for our method, just tune $\lambda$ for $L_{con}^u$, which generalizes well on all datasets. * Overall, just keep the weights of old and new loss to 1, and the weight for $L_{con}^u$ be slightly less than 1 (e.g, 0.7), our methods consistently achieves SOTA on all datasets. As a result, the hyper-parameter tuning is relatively simple. **References:** [R1]. Self-organization in a perceptual network. Ralph Linsker. 1988. Thanks for your feedback. If you still have questions, please feel free to continue the discussion.
Summary
The paper proposes a novel method for the Continual Generalized Category Discovery task, addressing the challenges of discovering new classes and preventing forgetting. The approach introduces Clustering-guided Initialization and Group-wise Soft Entropy Regularization for class discovery, as well as Hardness-aware Prototype Sampling for mitigating forgetting. Rigorous experimentation across several datasets demonstrates a significant improvement in performance.
Strengths
1. The organization, presentation, and writing of the paper are very clear, and the figures are attractive and easy to understand. 2. The paper analyzes and proposes solutions for several important issues in Continual Generalized Category Discovery (C-GCD), demonstrating strong innovation. 3. The experimental analysis is thorough, with excellent results. The sufficient ablation studies confirm the contribution of each proposed innovation.
Weaknesses
1. In the C-GCD setting described in the paper, during the continuously discovering stage, the samples of old classes are available. Why not use these old class data directly instead of the Hardness-aware Prototype Sampling method? Some explanation or experimental validation should be added. 2. How is the margin probability computed in line 173 if all the labels are not available? 3. The proposed method uses DINO pretrained ViT-B/16 as the backbone, which, as I understand, is pretrained on the ImageNet dataset. This results in information leakage for CIFAR-100, ImageNet-100, and TinyImageNet datasets in the continual learning process, as all classes are already known in the pretrained model.
Questions
Please see the Weaknesses
Rating
6
Confidence
5
Soundness
4
Presentation
4
Contribution
3
Limitations
No
Summary
The paper points out two-bias issues in CGCD: prediction bias in probability space and hardness bias in feature space. To tackle those two issues, they propose cluster-guided initialization and soft entropy regularization to mitigate prediction bias, and they propose hardness-aware prototype sampling to mitigate hardness bias and forgetting.
Strengths
* The paper is easy to follow. I really appreciate this writing idea: explain the problem and solve the problem. * The experiment show great improvement over existing method.
Weaknesses
1. Overclaim. The paper argues that they extend CGCD to realistic scenarios. But from the experiment setup, I am not convinced that the scenarios they consider are more realistic than existing work [16,18]. The stage and new classes are still limited. 2. Too many hyper-parameters and loss terms. There are too many hyper-parameters to adjust different loss weights, hindering its generalization. In addition, there are too many loss terms, hindering the understanding of the method. For example, what is the effect of the proposed Eq 4? 3. In Table 5, why KD improve New class by a large margin? What does the new class mean? The total novel class or the new class in the last task? 4. As the paper points out two issues: prediction bias and hardness bias, and proposes methods to mitigate the two issues, they should provide evidence to show that two issues are mitigated. Minor: 1. The rightmost one in Figure 2 is (d) instead of (b). 2. $L_{hap}$ in eq 10 and 11 should be consistent. 3. citation 23 is a little strange. It seems that it was published in ICCV2023. The citation should be corrected.
Questions
See weaknesses, especially 2 and 4. I will raise my score if the 2 and 4 are well resolved.
Rating
5
Confidence
4
Soundness
3
Presentation
4
Contribution
2
Limitations
None
Summary
The article presents a method for Continual Generalized category discovery. A de-biasing learning framework for the Category Discovery (C-GCD) task is designed to address the challenge of continuously discovering new concepts in an ever-changing environment while maintaining recognition of known categories. Traditional C-GCD studies have some limitations, such as storage and privacy issues caused by storing samples of past classes, and only considering limited incremental stages or assuming a proportion of known samples, which are not in line with practical application scenarios. Therefore, the study focuses on a more realistic C-GCD setup that includes more learning phases, more new categories, and after each phase, data from previous phases is not accessible.
Strengths
Originality: In this paper, the authors propose a novel de-biased-learning framework, "Happy, "specifically for the continuous Generalized Category Discovery (C-GCD) task, which is a relatively underexplored research area. It has designed a unique set of methods to handle the challenge of incrementally discovering new categories in unlabeled data while preserving the ability to recognize old categories, which traditional machine learning and deep learning approaches have overlooked. In particular, the proposed cluster-guided initialization, soft entropy regularization and hardness-aware prototype sampling strategies are innovative solutions to the unique problems of C-GCD. Quality: The quality of the paper is reflected in its detailed theoretical analysis and experimental verification. The authors not only elaborate the design principle and motivation behind the method, but also conduct extensive experiments on multiple datasets to prove the effectiveness of the "Happy" framework achieving significant performance improvement, showing its strong generalization ability and practicality. In addition, the paper also reflects on the limitations and assumptions of the method, indicating that the author has deeply considered the comprehensiveness and rigor of the study. Clarity: The structure of the paper is clear and the logic is coherent. From the introduction, which clearly explains the research background and motivation, to the method, which analyzes the components of the framework and their working principles in detail, to the presentation and analysis of the experimental results, every step is well organized. In addition, the authors also provide detailed implementation details and algorithm flow in the appendix, which increases the readability and reproducibility of the paper.
Weaknesses
1. Although the authors acknowledge and briefly discuss the social impact of technology, the paper does not detail specific negative impacts that a "Happy" framework could bring, such as potential bias transmission, fairness issues, privacy violations, or security risks. For an algorithm intended for application in the open world, the lack of a comprehensive social impact analysis may limit its acceptance and ethical application in practice. 2. It is clearly stated that this study does not include theoretical results, which means that a complete hypothesis set and theoretical proof are not provided to support the validity of the proposed method. While empirical research has shown the effectiveness of "Happy, "the lack of a solid theoretical foundation may diminish its persuasive power in academia. 3. The authors acknowledge that there are some assumptions and limitations to the study, such as overfitting the model, performance in noisy environments, and testing on specific datasets. These factors may limit the method's general applicability and ability to generalize, especially in different data distributions or more complex real-world environments. 4. It is pointed out that even after the introduction of the debias mechanism, the recognition accuracy of the new class still fluctuates at different incremental stages, which indicates that the model may have stability problems when processing data of different class difficulty. This volatility can affect model reliability and user trust in real-world deployments.
Questions
1. In the experimental part, how do you make sure that in the unsupervised increment phase, the model not only finds new categories, but also accurately distinguishes them without confusing them with the old ones? Are there specific metrics or experimental Settings to measure this ability to differentiate? 2. How does the soft entropy regularization realize the reasonable distribution of the probability of the new class? Can you explain this process in detail and how it is combined with cluster-guided initialization to improve the clustering performance of new categories? 3. In the hardness-aware prototype sampling strategy, how do you define and quantify the "hardness" of a class, and how do you sample effectively against this hardness to mitigate forgetting? Are there concrete examples of how this strategy helps the model remember old categories that are difficult to classify? 4. The paper mentions that in preliminary experiments it was found that the model tends to misclassify new categories into old ones, and that the features of the old categories are disturbed when learning the new ones. In addition to the proposed solutions, have other technologies, such as meta-learning or memory enhancement networks, been considered to further improve these problems?
Rating
5
Confidence
5
Soundness
3
Presentation
3
Contribution
3
Limitations
The authors discussed the limitations of the work in the paper, including the confidence calibration problem and the scope of the study mainly focused on the classification task. Confidence Calibration problem: In the Continuous Generalized Category Discovery (C-GCD) task, the confidence of the model is not calibrated due to an imbalance in the labeling conditions between the initial phase and the sustained phase. This leads to a clear gap between the old category and the new category, and even degrades performance when incorporating prior information. The scope of application is limited to classification tasks: Although this paper mainly discusses the C-GCD learning paradigm under classification tasks, the application of this method has not been extended to other fields, such as object detection and image segmentation. The authors note that future work should consider how confidence calibration techniques can be incorporated into C-GCD to further reduce potential bias. At the same time, they encourage the extension of the C-GCD learning framework to more types of visual tasks in order to expand its applicability and impact. These discussions reflect a clear understanding of the limitations of the current findings and suggest possible directions for future research.
Thank you for your response. Some of my concerns have been addressed. However, regarding W2, the author may have misunderstood my question. What I want to know is how to determine whether a sample belongs to an old class or a new class within a session when all labels are unavailable. The marginal probability for old and new classes can only be computed individually if there is a way to accurately separate old class samples from new class samples in each session in an unsupervised manner.
Response to Reviewer jUDg
Dear Reviewer jUDg, Thank you very much for your feedback. Here, we provide a detailed explanation of W2: how to compute the marginal probability: * In this paper, marginal probability for old $\overline p_{old}$ and new classes $\overline p_{new}$ refer to the sum of $\overline{\boldsymbol{p}}\in R^{K}$ (marginal probability of all samples in a batch) at the indices of new and old classes, which is calculated **along the class dimension**, **rather than along the sample dimension**. * Here both $\overline p_{old}$ and $\overline p_{new}$ are **scalars**, while $\overline{\boldsymbol{p}}\in R^{K}$ is a K-dim **vector**, $K=K_{old}+K_{new}$ denotes the number of total classes of the current stage. * As a result, **we do not need to separate old class samples from new class samples when we compute $\overline p_{old}$ and $\overline p_{new}$**. * Specifically, for a batch of samples, let $\boldsymbol p_i$ denotes the prediction of the i-th sample. We first average the model predictions for **all samples** along the batch dimension, **without distinguishing between new and old classes samples**, namely $\overline{\boldsymbol{p}}=\frac{1}{|B|}\sum_{i\in B}\boldsymbol p_i$. Here, both $\overline{\boldsymbol{p}}$ and $\boldsymbol p_i$ are $K$ dim vectors. Then we calculate the marginal probabilities for old and new classes **along the class dimension**, i.e., $\overline p_{old}=\sum_{c=1}^{K_{old}}\overline{\boldsymbol{p}}[c]$ and $\overline p_{new}=\sum_{c=K_{old}+1}^{K}\overline{\boldsymbol{p}}[c]$ (so both $\overline p_{old}$ and $\overline p_{new}$ are scalars and $\overline p_{old}+\overline p_{new}=1$). Note that $\overline{\boldsymbol{p}}[c]$ represents the c-th index of the vector $\overline{\boldsymbol{p}}$. * **Here is one example**. On the first stage of CIFAR100, there are 50 old classes and 10 new classes, so the dimension of model predictions is $K=60$. We first compute the average of model predictions for all samples in a batch and obtain the **vector** $\overline{\boldsymbol{p}}$. Then we sum the first 50 dimensions of $\overline{\boldsymbol{p}}$ to obtain the **scalar** $\overline p_{old}$ and sum the last 10 dimensions to obtain the **scalar** $\overline p_{new}$. * To conclude, $\overline p_{old}$ and $\overline p_{new}$ are marginal probabilities of **all** samples **along the dimension of old/new classes**, rather than the marginal probability of **old/new samples**. So we do not need to determine whether a sample belongs to an old class or a new class. We really appreciate your feedback. If you have any additional questions, please let us know. We greatly respect your insights and enjoy the discussions with you. Thanks again!
I read all the comments of the other reviewers. I agree with all weaknesses of the other Reviewers. I hold the same opinions.
Dear Reviewer TDQJ, Thanks very much for your time and valuable comments. In the rebuttal period, we have provided detailed responses to all your comments and questions point-by-point for the unclear presentations. Any comments and discussions are welcome! Thanks for your attention and best regards, Authors of Submission 5010.
Dear Reviewer TLvc, Thanks very much for your time and valuable comments. In the rebuttal period, we have provided detailed responses to all your comments and questions point-by-point for the unclear presentations. Any comments and discussions are welcome! Thanks for your attention and best regards, Authors of Submission 5010.
Thanks to the author for answering my question, but it failed to solve my doubts about the weakness of the method
Further Response to Reviewer TDQJ
Dear Reviewer TDQJ, Thank you very much for your feedback. Regarding the weaknesses you've highlighted, we have further organized our responses and summarized them as follows: * W1: Lack of a comprehensive social impact analysis. * Here, We have included a more detailed discussion on societal impacts, supplemented with specific examples. * **Medical risks:** Our method could be applied in the medical field, where biases in learned categories might affect the discovery or diagnosis of new diseases. Over time, these errors could increase, potentially delaying critical medical interventions. * **Fairness issues:** Regarding different genders and populations, our model may learn unfair biases from the labeled data, which could then be transferred to new domains. This could perpetuate inequalities within the model's predictions. * W2: Lack of a theoretical foundation. * The theoretical foundation of our method is the **InfoMax principle** [R1], namely **maximizing the mutual information (MI) between the inputs and outputs of a system.** Here inputs and outputs denotes the feature $Z$ and model predictions $Y$. * For the self-training problem of learning new classes in C-GCD, we employ the maximizing the mutual information (MI) theory. Specifically, the objective can be expressed as $\max I(Y, Z) = H(Y) - H(Y|Z)$, where $Y$ represents the labels and $Z$ the features, with $I$ and $H$ denoting mutual information and entropy, respectively. * The proposed entropy regularization on model predictions $Y$ is the minus of first term $-H(Y)$, while the self-distillation objective aims to encourage the model to predict more confidently, resulting in lower prediction entropy, namely the second term of minimizing $H(Y|Z)$, namely maximizing $-H(Y|Z)$. * Overall, our method follows the Infomax principle and aims to maximize mutual information, to ensure the quality of self-learning. * W3: There are assumptions that may limit the general applicability, especially in different distributions. * We need to clarify here that our method does not rely on explicit assumptions or predefined ratios of new to old categories in previous methods, thereby enhancing its generalizability. This has been demonstrated in our experiments, including the distribution shift experiments detailed in the Rebuttal PDF. * W4: The accuracy of the new class still fluctuates, which can affect model reliability and user trust in real-world deployments. * Our method significantly mitigates fluctuations. For example, in CIFAR 100, the gap in accuracy among classes has been reduced from 17.3% to 6.3%. This ensures the reliability and stability of our approach in real-world deployments. **References:** [R1]. Self-organization in a perceptual network. Ralph Linsker. 1988. Hope these further responses address your concerns about the weaknesses part. If you have further questions, please let us know. We are willing to have more discussions with you. Thanks for your participation.
Thanks for your recognition
Dear Reviewer jUDg, We are pleased that our responses have addressed your concerns. Thank you very much for your insightful suggestions and valuable efforts, which are crucial for enhancing the quality of our paper. Thank you once again.
Further Response to Reviewer TLvc
Dear Reviewer TLvc, We are glad that our responses have resolved your concerns. Regarding the issues **W2 and W4**, which you are particularly concerned about, we have further organized our responses, summarized as follows: * W2: Too many hyper-parameters and loss terms. * **Understanding of the method.** Our method consists of three parts: $L_{Happy}=L_{new}+L_{old}+\lambda L_{con}^u$. 1. Cluster new classes: $L_{new}$, with self-training and mitigation of prediction bias for new class. 2. Mitigate forgetting old classes: $L_{old}$, with hardness-aware prototype sampling to alleviate hardness bias. 3. Contrastive learning: $L_{con}^u$ to generally ensure feature representations. * **Hyper-parameter tuning.** For $L_{new}$ and $L_{old}$, our results show that a 1:1 weight ratio is optimal, because both old and new classes matter and we need to maintain a balanced focus on both new and old classes as a whole (other weight ratios of $L_{new}$ and $L_{old}$ will result in a 5% to 10% performance degradation). So keep the weight ratio of $L_{new}$ and $L_{old}$ to 1, and the only parameter for tuning is $\lambda$ for $L_{con}^u$. We found that $\lambda$ equals 0.7$\sim$1 works fine. Therefore, It is simple to choose hyper-parameters for our method, just tune $\lambda$ for $L_{con}^u$, which generalizes well on all datasets. * **Overall.** The three objectives are complementary and non-conflicting, where each is essential, and they collectively improve classification across all classes. * W4: Provide evidence to show that two issues are mitigated. * Here, we provide more intuitive examples. * **Prediction bias** refers to the bias that the model tends to predict new class samples to the old ones. **Metric:** We compute the proportion of new classes' samples misclassified as old classes. **Example:** In our method, the ratio of new samples misclassified to old ones decreases from 63.25% to 10.20%, which means that Happy could largely mitigate prediction bias, with better separation between old and new classes. * **Hardness bias** refers to the bias that models have weaker classification and more severe forgetting on more hard old classes. **Metric:** We compute the accuracy of the hardest class and the accuracy variance among all old classes. **Example:** On CIFAR100, our method improves the hardest accuracy from 65.10% to 70.23%, and the variance decreases from 23.04% to 10.33%, which means our method greatly mitigates the forgetting of hard old classes and alleviates the extent of imbalance caused by hardness bias. We hope our further responses regarding **W2 and W4** have clearly resolved your concerns. We were wondering whether our paper could be re-evaluated considering these further explanations on **W2 and W4**. If you have any other questions, we would be happy to further discuss with you. We really appreciate your feedback.
Decision
Accept (poster)