DenoiseRep: Denoising Model for Representation Learning

Summary

This paper proposes DenoiseReID to improve feature discriminative with joint feature extraction and denoising, in which FEFDFA is developed to merge parameters of the denoising layers into embedding layers. Experimental results show the proposed DenoiseReID improves performance.

Strengths

1. The proposed joint representation learning and denoising process. 2. The proposed FEFDFA is a computation-efficient algorithm.

Weaknesses

The author solves all my quentions.

Questions

N/A

Rating

8

Confidence

5

Soundness

4

Presentation

4

Contribution

4

Limitations

Yes

Summary

This mauscript proposes a novel denosing model for representaetion learning and take person re-identification as a benchmark. It unifies the frameworks of feature extraction and feature denoising, where the former progressively embeds features from lowlevel to high-level, and the latter recursively denoises features step-by-step. Besides, a FEFDA is proposed to fuse feature extraction and denoising in a single backbone without changing its structure and taking extra runtime latency. Experiments on ReID, large-scale and fine-grained classification tasks show its effectivenss.

Strengths

1. The idea of unifying feature extraction and feature denoising in a single backbone without changing its structure is interesting. As far as I know, its first time to see the idea. 2. The characteristic computation-free and label-free is promising. The thoerical analysis seems precies and right. 3. Experiments on 3 typical tasks and 9 datasets of representation learning (retrieval, classfication, fine-grained classification) are sufficient and extensive.

Weaknesses

1. the proposed "Feature Extraction and Feature Denoising Fusion Algorithm" is little similar to reparameterization, please clarify their difference. 2. its application to transformer series are well analyzed, it will be better if peformance on CNN series are shown. 3. representation learning is a wide and fundational conception, its applications on more downstream tasks, such as detection, segmentation even generation, could be analyzed, in future. 4. The hyper-parameter analysis is missed. 5. This method seems need extra training steps, what if the baseline methods are trained under the same steps?

Questions

For questions, please see the weakness in the above section.

Rating

8

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have discussed the limitations of the proposed method.

Summary

This paper proposes a new method, Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA), which utilizes the denoising ability of diffusion models to denoise the features in the feature extraction layer, and fuses the parameters of the denoising layer with those of the feature extraction layer through parameter fusion, further improving retrieval accuracy without incurring additional computational costs. The effectiveness of the FEFDFA method has been validated on multiple common image discrimination task datasets.

Strengths

1.The article structure is complete and writing is generally clear. 2.Some experimental results seem to good.

Weaknesses

1.In fact, the intermediate layer features of the pre-trained diffusion model can be used directly for the downstream task such as the discrimination task [1][2][3]. The authors need to enrich the Related Work. 2.Line144-146, no evidence provided to support the proposed hypothesis. 3.The authors treat the backbone as a series of denoising layers, so the training loss in Eq. (11) should be the sum of the MSE losses of each denoising layer. 4.The statement is inconsistent. Line172-173，"we freeze the original parameters and only trained the FEFDFA." Line509-511，"the parameters of the FEFDFA and baseline were trained alternately." In addition, unless the baseline and FEFDFA are trained together, Eq. (12) is incorrect. [1] Mukhopadhyay, Soumik, et al. "Diffusion models beat gans on image classification." arXiv preprint arXiv:2307.08702 (2023). [2] Li, Alexander C., et al. "Your diffusion model is secretly a zero-shot classifier." ICCV 2023. [3] Baranchuk, Dmitry, et al. "Label-Efficient Semantic Segmentation with Diffusion Models." ICLR 2022.

Questions

same to the weaknesses.

Rating

6

Confidence

5

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors have addressed the limitations.

Summary

This paper proposes a novel denoising model called DenoiseReID, designed to enhance representation learning in person re-identification (ReID) tasks. This approach combines traditional denoising processes with feature extraction through a feature extraction and denoising fusion algorithm (FEFDFA) that incurs no additional computational cost. It incrementally improves the discriminability of features without the need for labels.

Strengths

The experimental results demonstrate that DenoiseReID achieves stable performance improvements across multiple ReID datasets. Furthermore, it can be extended to large-scale image classification tasks such as ImageNet, CUB200, Oxford-Pet, and Flowers datasets.

Weaknesses

1. In the comparative experiments, the authors missed an opportunity to benchmark their method against the latest advancements like CLIP-ReID, restricting comparisons solely to TransReID. 2. When contrasted with TransReID, this approach shows a moderate enhancement in performance.

Questions

I am particularly concerned about the performance comparison in this paper. Is it necessary to use diffusion models for conducting ReID tasks, and what advantages do they offer compared to CNNs or ViTs? Currently, it seems that their performance improvement is rather modest.

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

yes, the authors adequately addressed the limitations.

Open Discussion

Hi dear reviewers, we thank all your professional and kind comments. We have carefully read and responded to all the comments. Please allow me to clarify some misunderstandings and highlight our contributions. Our proposed method is test-time computation-free (FREE LUNCH!) and scalable to various baseline and downstream tasks, which has been proven in CNN series (e.g. ResNet50), Transformer series (e.g. ViT, Vmanba), Person Re-ID (MSMT, Duke), Vehicle Re-ID (vehicReID), Large-Scale Image Classification (ImageNet), Fine-Grained Image Classification (CUB) and Object Detection (COCO). For its scalability, we DO NOT hack it for backbones or tasks. With the same hyper-parameters, ours achieve consistent and stable improvement on ALL backbones and tasks above. For example, based on the famous object detection method Mask-RCNN, our proposed method achieves extra improvement from 1.6%-1.1% on COCO. Based on previous state-of-the-art CLIP-ReID, our proposed method achieves extra improvements from 0.8%-0.6% on Duke, MSMT and Market. All those improvements take NO extra test-time computation and NO specific hyper-parameters customized by tasks. Considering the advantages of **test-time computation-free** and the **consistent and stable improvement on the MANY backbones and tasks** (CNN series, Transformer series, Person ReID, Vehicle ReID, Occluded ReID, Large-Scale Image Classification, Fine-Grained Image Classification, Object Detection), we believe the proposed method will make significant contributions to the Representation Learning community. Code will be released in future.