Summary
This paper addresses the On-the-fly Category Discovery (OCD) task, which involves utilizing existing category knowledge to recognize both known and unknown categories in new data streams in real-time. To tackle the high sensitivity and suboptimal feature representation issues of existing methods when dealing with fine-grained categories, the paper proposes an innovative Prototypical Hash Encoding (PHE) framework. This framework uses a Category-aware Prototype Generation (CPG) module to represent each fine-grained category with multiple prototypes and employs a probabilistic masking strategy to encourage the model to fully capture intra-class diversity. The Discriminative Category Encoding (DCE) module maps the generated category prototypes to low-dimensional hash centers, optimizing image hash features to ensure intra-class compactness and inter-class separation. Additionally, a center separation loss function is designed to maintain a minimum Hamming distance between different category hash centers. Experimental results on multiple datasets confirm the superiority of this method.
Strengths
The proposed PHE method demonstrates its superiority over existing state-of-the-art methods across multiple datasets. It effectively addresses the issues of large intra-class variance and small inter-class variance, while minimizing the information loss associated with dimensionality reduction. The paper employs visualization techniques to analyze the underlying mechanism by which PHE groups samples into known or unknown categories. Moreover, due to its optimization based on hash centers and inference process based on Hamming balls, the PHE method shows stable performance across different hash code lengths, effectively mitigating the "high sensitivity" problem. In contrast to the SMILE method, which exhibits significant accuracy degradation and instability with increasing hash code lengths, the PHE method maintains remarkable stability and consistency.
Weaknesses
Although the proposed Prototype Hashing Encoding (PHE) framework outperforms existing methods in terms of performance, further research is needed to improve its accuracy in recognizing unknown categories. Additionally, due to the need to compute multiple hash centers and perform complex distance calculations, the computational cost of PHE is relatively high, especially on large-scale datasets.
Questions
Apart from the issues mentioned in weakness part, there are some additional questions regarding implementation details:
1. The article mentions that to encourage the model to fully capture intra-class diversity, K categories are equally allocated to each class. I would like to know how significant the choice of K is on the model's performance, and if there are experiments demonstrating the impact of different K values on the model.
2. Similarly, when using the masking strategy, how does the variation in the value of θ affect the training and final performance of the model?
3. What about the performance of different splits of old and new class, further analysis is needed regarding the size of different splits as well as whether the correlation of old and new classes essentially influence the final performance etc. Some in-depth insights are expected.
Limitations
As mentioned in the appendix of the manuscript, although the proposed PHE framework outperforms existing methods in terms of performance, further research is needed to improve its accuracy in recognizing unknown categories. Additionally, the PHE method involves multiple hyperparameters that require careful tuning to achieve optimal performance, increasing the complexity of model debugging and optimization. Moreover, the weights of the component loss functions in the total loss function need to be appropriately set to balance the optimization objectives of different components, which may require extensive experiments to find the suitable weight combination.