Goal Conditioned Reinforcement Learning for Photo Finishing Tuning

Dear Reviewer WirW, In our rebuttal, we test our model on an unseen dataset, HDR+. Results demonstrate that our method generalizes well to unseen datasets, outperforming all baselines by a large margin. We also explained that the CMAES baseline is implemented with a multi-core CPU with parallel acceleration. We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed. Thank you again for your time and effort.

Dear Reviewer kK87, In our rebuttal, we address the implementation details of three baselines, explain the code used, and clarify terms in the paper, along with citations of CMAES (`Q1`). We also point out that certain behaviors of the baselines (such as the similar hue shift and green image in Figure 4) are due to the inherent flaws of the baselines (`Q2` `Q3`). Additionally, we provide more detailed experiments of the baselines to prove the correctness of our reproduction (`Q4`). We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed. Thank you again for your time and effort.

Dear Reviewer syDC, In our rebuttal, we explain the reason for choosing TD3 and include further ablation studies to show that the choice of RL algorithm does not significantly impact performance. We also conduct more ablation over the greedy algorithm as suggested by the reviewer. Additionally, we test our method on the HDR+ dataset. Results demonstrate that our method generalizes well to unseen datasets. We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed. Thank you again for your time and effort.

Dear Reviewer syDC, Thank you for your reply. We would like to further discuss the questions with you below: ### Q1. why TD3 performs better In our task, the RL algorithm is only a tool for optimizing the policy network. We analyze why TD3 performs better as follows: TD3 is an off-policy RL algorithm with a deterministic policy. The deterministic policy allows it to explore the entire action space, including actions near the boundary, which is crucial for recovering boundary ISP parameters in our photo finishing task. TD3 also employs tricks like target policy smoothing and delayed policy updates to enhance robustness. In our task, actions like significantly brightening an image can cause large changes in the reward function, so these robustness tricks are crucial to the stability of the RL algorithm. In contrast, SAC uses a stochastic policy, sampling actions from a Gaussian distribution, which reduces the likelihood of exploring boundary actions, impacting performance. PPO also faces this issue with its stochastic policy. Both SAC and PPO also lack robustness tricks like target policy smoothing. Moreover, PPO is an on-policy algorithm, making it less sample-efficient in our complex photo finishing environment, potentially leading to suboptimal results. | | deterministic policy | robustness tricks | off-policy | | ---- | -------------------- | ----------------- | ---------- | | TD3 | ✔ | ✔ | ✔ | | SAC | ✗ | ✗ | ✔ | | PPO | ✗ | ✗ | ✗ | These differences explain the variation in performance, but the effect of the RL algorithm is still minor compared to state representation. As shown in the first table of the rebuttal, TD3, SAC, and PPO all yield a PSNR of around 37 **with state representation**, whereas TD3 without state representation achieves only 32.17. ### Q2. multiple seeds We set the random seed to 1 in all our experiments in the paper and the rebuttal, ensuring fair comparison. In our main comparisons with all baselines, our method outperforms them by a large margin (over 4 dB in PSNR). Even in the ablation study over different RL algorithms, TD3 still outperforms SAC by 1.16 dB in PSNR. As discussed earlier, the RL algorithm is just a tool within our framework, and we are confident that we have selected the RL algorithm that best fits our task. Additionally, we plan to include the average scores over multiple seeds in the camera-ready version of this paper. ### Q3. greedy tuning performs well & stronger baseline with conditioning **There are no stronger baselines available.** We followed the setup from [27] for baselines in the photo finishing task. We compared our method to existing work in the field of photo finishing tuning, including CMAES [18] and proxy-based methods [26, 27]. Currently, there are no stronger baselines for the photo finishing tuning task. **Search-based methods (Greedy & CMAES) perform well because they optimize over more iterations.** Both the greedy tuning algorithm and CMAES are search-based methods. The search-based methods, including the greedy algorithm added in the rebuttal, only performs well when using 200 searching iterations, while our method can achieve better results with only 10 iterations. The main advantage of our RL method is its fast convergence speed, as shown in Table 2 in our paper. **Conditioning is not possible for greedy algorithm and is part of our contribution.** Traditional search-based methods, including greedy algorithms, are driven purely by the loss function's output relative to the parameters being adjusted. These methods lack the ability to condition on complex features like the characteristics of the input image, the target image, or prior processing operations. Such conditioning requires an understanding capability and the ability to generalize from past experiences, which is beyond the capabilities of traditional search-based algorithms. Developing a novel search algorithm conditioned on these complex characteristics is one of our contributions. Our proposed state representation encodes information about the input image, target image, and prior image processing operations. The policy network can then leverage this information to make decisions that are conditioned on these inputs, which leads to more efficient and effective optimization. ### Q4. why the proposed method generalizes well We have analyzed why our method generalizes better in Q1 (cross-dataset generalization) of the **To All** section in our rebuttal.

Dear Reviewer ASBs, In our rebuttal, we explain whether each baseline requires training. We also provide more evaluation on the HDR+ dataset. Results demonstrate that our method generalizes well to unseen datasets, outperforming all baselines by a large margin. We want to follow up to see if our responses address your concerns. We would be very grateful to hear additional feedback from you and will provide further clarification if needed. Thank you again for your time and effort.