Test-Time Dynamic Image Fusion

Summary

This paper introduces a method for dynamically adjusting fusion weights for image pixels based on their relative dominability, calculated using pixel-wise reconstruction losses. This approach aims to minimize generalization error by considering the correlation between fusion weights and reconstruction losses.

Strengths

1- The presentation of the method is simple and clear. 2- As the results show, the proposed framework achieves good results on several datasets.

Weaknesses

1. The claim that RD accurately captures the dominant regions of different sources without solid empirical justification for various scenarios might be overreaching. It is unclear how RD performs under various noise conditions or with sources of different qualities. 2. In Eq. (4), the fusion weights $w$ are important as they determine the contribution of each source to the total loss. However, there is no clear definition for normalizing these weights. 3. In Eq. (8), the paper lacks discussion on the initialization of $w^(m)$ and how it affects convergence and stability during the dynamic adjustment process. 4. Figure 2 is not clear. How are the feature maps of different layers fused, and what is the impact of each of them on the overall performance of the proposed model?

Questions

Please see the weaknesses!

Rating

5

Confidence

3

Soundness

2

Presentation

2

Contribution

2

Limitations

Not very adequate. The limitation is general.

Summary

This paper proposes a theoretical justification of image fusion from a generalization perspective and reduces the upper bound of generalization error by decomposing the fused image into multiple components corresponding to its source data. A new test-time dynamic image fusion paradigm TTD is further proposed with the finding that the negative correlation between fusion weight and the uni-source reconstruction loss is the key to reducing the generalization loss. Extensive experiments and discussions confirm the theory and superiority.

Strengths

The idea that applying the test-time adaption method into image fusion task with theoretical guarantee is quite meaningful and experiments are sufficient.

Weaknesses

The details presentation and explanation about test-time adaption are not very clear.

Questions

1. Is the idea that adding up every uni-source data linearly to get fused image reasonable enough? Are there any information only found with several data neglected in the whole process? 2. Mathematical formulas in Appendix A.1 are a little confused. More parentheses ought to be used to present clear explanations about the scopes of every mathematical symbols. 3. Is the TTD applied to every combination from sources? Most test-time adaption methods only need few data to fine-tune the model. But this paper seems to apply TTD to all source data.

Rating

7

Confidence

5

Soundness

3

Presentation

2

Contribution

3

Limitations

This paper presents a new perspective to analysis the generalization error of the image fusion task, the details presentation and explanation are not very clear, mathematical formulas are little confused and more explanation about inference process should be provided.

Summary

This paper proposes a theoretically guaranteed new paradigm for test-time dynamic image fusion, which exploits the negative correlation between the fusion weights and the single-source reconstruction loss to reduce the upper bound of the generalization error. Extensive experiments demonstrate its effectiveness on a variety of image fusion tasks.

Strengths

The proposed method is simple and effective，the experimental results are detailed and rich, and the effect is competitive compared to SOTAs.

Weaknesses

1. Authors does not mention the change in model inference efficiency. 2. The paper assumes that the decoder of the fusion model is a CNN model, which introduces certain limitations. If the model were a Transformer or Diffusion model, would TDD still be effective? 3. For specific fusion tasks, it is recommended to compare specific methods rather than generalized methods, e.g., multifocus image fusion tasks should compare multifocus image fusion methods. 4. Ablation lacks enough persuasiveness. 5. Table I and II are written inconsistently e.g. TDD and Ours.

Questions

See Weaknesses.

Rating

7

Confidence

5

Soundness

3

Presentation

2

Contribution

3

Limitations

See Weaknesses.

Summary

- This paper tries to solve the image fusion task, where multi-source images are provided and one needs to extract and integrate effective information from them. - The paper demonstrates its effectiveness on four different tasks: VIF, MIF, MEF, and MFF. - The paper proposes a test-time dynamic image fusion method with theoretical justification. - This paper theoretically proves the superiority of dynamic image fusion over static image fusion, and provides a generalization error upper bound. - By using the relative domainability of each source as the dynamic fusion weight, it is able to theoretically improve the generalization of the image fusion model and dynamically emphasize the dominant regions of each source. - This method theoretically and empirically demonstrates superiority over static fusion methods through extensive experiments on various datasets, including visible-infrared, medical, multi-exposure, and multi-focus image fusion tasks.

Strengths

- The paper is well written. - The method is evaluated on four different tasks. - The approach is fairly simple. - The approach is justified theoretically - When the baseline model is robust, it can improve the fusion performance.

Weaknesses

- The adaptation method heavily relies on the performance of the baseline model, which can be ineffective when the model performance is poor. - The improvement over the non-adaptive baseline is minor.

Questions

-

Rating

5

Confidence

3

Soundness

3

Presentation

4

Contribution

2

Limitations

-