Adaptive Domain Learning for Cross-domain Image Denoising

Summary

This paper introduce adaptive domain learning (ADL) for cross domain raw image denoising problem. The author also introduce a module to embed sensor information into the network. Experiments on public datasets with various smartphone and DSLR cameras demonstrate proposed model outperforms prior work on cross-domain image denoising with a small amount of image data from target domain.

Strengths

1. The proposed pipeline is straightforward and the idea is easy to understand. 2. It's interesting to apply domain adaptation idea for raw image denoising.

Weaknesses

1. Novelty Issue: Although I am reluctant to mention this, the proposed pipeline is quite common and widely available in domain adaptation research. The main credit is that the author applied it to raw image denoising. However, it may require more innovative insights to meet the standards of the NeurIPS community. 2. Embedding sensor information to help the network distinguish the domain is also a common technique used in many research works. I don't believe it can be considered a significant contribution. 3. In line 210, 20 pairs of data in the target domain is actually not a very small number for a raw image dataset. I am curious about the significance of using these target domain data and how it can contribute to real advancements in raw image denoising applications. 4. The author should fine-tune PMN (the recent SOTA method) with the target domain data for a more thorough comparison. 5. Minor Issues: There is a misalignment in Table 2. It would be better to note SSID, ELD, and SID in Table 1 instead of using camera names only. PMN (Learnability Enhancement for Low-Light Raw Image Denoising: A Data Perspective)

Questions

Primarily comparing PSNR with self-supervised denoising methods is somewhat unfair because the primary advantage of these methods is not high PSNR. Instead, why not structure the experimental section to compare with state-of-the-art performance methods?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

No

Summary

This paper proposes a novel adaptive domain learning (ADL) approach for cross-domain RAW image denoising problem. The ADL scheme allows to train models for target domains with limited data by leveraging data from other source domains. The harmful data from the source domain is automatically removing during training. Authors also propose a new modulation approach to encode the sensor type and ISO to help the network to adapt to different noise distributions. The extensive experimental results demonstrate the effectiveness of the proposed method.

Strengths

1. The proposed ADL training strategy is useful in the cases of very limited target domain data and is able to effectively utilize the existing source domains by filtering out harmful data 2. Extensive ablation studies demonstrate the robustness of the method even in the presence of misaligned data 3. The experiment results in the supplementary show that the method generalizes to other problems (dehazing, deblurring)

Weaknesses

1. The qualitative results only show the error maps. Actual denoised images (or patches) in the sRGB should be provided for better visual comparison of the methods. In addition, quantitative analysis with metrics such as LPIPS should be done on the sRGB methods 2. It seems that the already trained network cannot easily adapted to new data. The method encodes the sensor type as a one-hot vector. Therefore, the new source domain data (new sensor) cannot be incorporated to fine-tune the network and the training should be done from scratch.

Questions

Have you tried other metrics than PSNR for dynamic validation?

Rating

6

Confidence

5

Soundness

4

Presentation

3

Contribution

3

Limitations

None

Summary

This paper address the cross-domain image denoising problem with a small number of target domain training samples. The authors propose an adaptive domain learning (ADL) strategy that dynamically selects useful training samples from both source and target domains to improve performance. Additionally, the paper leverages channel-wise modulation layer to model different noise distribution caused by sensor type and ISO. Experimental results showcase the advantages of the proposed method.

Strengths

1. The paper clearly defines the problem of few-shot domain adaptation for image denoising and presents a solution. 2. The reported quantitative results in the paper indicate improved performance over existing methods, suggesting potential consideration and applicability in real-world scenarios. 3. In general the writing is clear and easy to follow, with minor presentation issues.

Weaknesses

1. **Lacking Clarity on Definitions and Practical Benefits:** The paper actually discusses the few-shot domain adaptation problem but fails to clearly define the specific settings and scenarios being addressed. Also, this paper does not clearly specify the conditions under which the proposed ADL strategy provides the most benefit. For example, how does the performance of ADL compare with simpler strategies like mixing the source and target domain data as the number of target samples increases? What is the threshold number of samples where ADL proves advantageous? Is this number large enough so that collecting target domain data larger than the threshold can be seen as impractical. 2. **Lack of comparison to adequate baselines**: The paper would benefit from a more comprehensive comparison with existing few-shot domain adaptation method in low level vision [1,2]. Comparing the proposed method with approaches in different settings, such as self-supervised learning, may not be fair or relevant. It would be more informative and meaningful to compare against methods with similar objectives and constraints. 3. **Lack of Task-Specific Design and Motivation**: The proposed ADL strategy is not specifically tailored for the denoising task, which reduces its effectiveness and relevance. Additionally, the paper does not clearly explain the motivation behind combining ADL with the modulation module. The paper would benefit from a clearer explanation of how these components interact and complement each other. 4. **Presentation Issues:** There are inconsistencies in the presentation of results, such as the caption in Table 3 mentioning "Mod" (modulation) without corresponding columns in the table. [1] K. Prabhakar, Vishal Vinod, et al. "Few-Shot Domain Adaptation for Low Light RAW Image Enhancement." British Machine Vision Conference 2021. [2] Bo Jiang, Yao Lu, et al. "Few-Shot Learning for Image Denoising." 2023.

Questions

1. As the size of the target domain dataset increases, how does the performance of the ADL strategy compare with simpler strategies such as mixing the source and target domain datasets? Is there a threshold number of target samples beyond which ADL does not offer significant benefits? 2. What are the computational overheads associated with the dynamic validation set used in the ADL strategy? Is the process efficient enough?

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

Limitations are currently not discussed in this paper, which may not be so adequate.

Summary

This paper addresses the challenge of cross-domain RAW image denoising due to varying noise patterns from different camera sensors (bit depths, color). The authors propose an Adaptive Domain Learning (ADL) scheme that leverages existing data from various sensors and a small amount of data from a new sensor to train an effective denoising model. The ADL scheme selectively uses beneficial data from the source domain while discarding harmful data, and introduces a modulation module to incorporate sensor-specific information. The proposed model outperforms prior methods on public datasets, demonstrating state-of-the-art performance with limited target domain data.

Strengths

1. Originality: The adaptive domain learning (ADL) strategy is a novel approach to addressing the cross-domain image denoising challenge. 2. Quality: The proposed method is thoroughly evaluated through extensive experiments on various public datasets, showing its robustness and effectiveness. 3. Clarity: The paper is well-structured, with clear explanations of the methodology, experiments, and results.

Weaknesses

1. Compared with RAW2RAW mapping: There are existing methods for raw-to-raw mapping, such as "Semi-Supervised Raw-to-Raw Mapping." It would be interesting to explore whether source domain raw images can be mapped to the target domain and then used for training based on the noise model. This could potentially provide a more direct comparison and integration of data from different domains. 2. Data Requirements: The authors used over 20 captured raw images, which is significantly more than the "Two pairs" approach used in LED. With these raw images, could a noise model be used to simulate noise instead? Is there a comparison available? This would help to understand whether the additional raw images provide a significant advantage over using simulated noise from a noise model.

Questions

The current ADL settings require more data compared to the LED method. In Table 2b, it is not clear how the LED method was specifically trained—whether it involved fine-tuning with limited target domain data. Additionally, comparing the performance of ADL with LED under the "Two pairs" condition would provide valuable insights into the effectiveness and efficiency of both approaches. This could help understand the trade-offs between data requirements and performance outcomes.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

A more explicit discussion on the potential impact of extremely limited target domain data (like two pairs) on the model's performance would provide additional clarity.