Binarized Spectral Compressive Imaging

Summary

This paper presents a deep neural network with binarized operations for low-cost spectral compressive imaging of hyperspectral images (HSI). The network introduces a basic unit for model binarization which adaptively redistributes HSI representations before binarization activation and uses a scalable hyperbolic tangent function to approximate the Sign function in backpropagation. Four binarization convolution modules are designed to solve the dimensional mismatch problem during feature reshaping and propagate full precision information throughout the network. Experiments on both synthetic and real data are conducted for performance evaluation.

Strengths

1. An effective binarized deep neural network is proposed for HSI reconstruction. 2. Four binarized convolution modules are designed to enable the propagation of full precision information through all convolution layers. 3. Based on the ablation experiments in the article, the proposed scalable hyperbolic tangent function provides inspiration for approximating the symbolic function in back propagation.

Weaknesses

1. The proposed binarization scheme is general, without specific design optimized for HSI restoration. Then, it lacks comparison to other general binarized NNs or comparison in other reconstruction tasks. 2. The proposed scheme is quite straightforward and engineering. 3. Approximation error bounds are not discussed, but this is quite important for binarized NNs. 4. Some key technical parts are not clearly/detailly described.

Questions

1. The paper argues that, as mobile devices are more and more widely used, the demands of running and storing HSI restoration models on edge devices grow significantly. Could the authors provide practical cases where mobile edge devices are used for HSI restoration? 2. In the binarized convolution layer, the formula for converting the model parameters from 32 bits to 1 bit is not clearly written. the average of the 32-bit absolute value multiplied by the sign function cannot be turned into a 1-bit number. 3. It is not clear how the proposed method deals with the dimensional mismatch feature remodeling process, and how to solve the dimensional mismatch with binarized convolution modules.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Approximation error bounds of the proposed binarization scheme are not discussed.

Summary

This paper proposes a novel method, Binarized Spectral-Redistribution Network (BiSRNet), for efficient and practical HSI restoration from compressed measurement in snapshot compressive imaging (SCI) systems. This paper redesigns the base model and presents the basic unit (BiSR-Conv) for model binarization. Specifically, this convolutional layer of BiSRNet is tailored for hyperspectral image processing. Comprehensive quantitative and qualitative experiments show that the proposed BiSR-Net outperforms state-of-the-art binarization methods and achieves comparable performance with full-precision algorithms.

Strengths

(1) This paper redesigns a U-Net consisting of the four BCMs as shown in Figure 4 to let the full-precision information flow pass by the all network. This point is critical since previous BNNs do not consider all the situations in feature reshaping. Then in the basic unit, BiSR-Conv, with the insight of treating different HSIs with different densities and distribution, this paper proposes to shift HSIs before binarization to allow more binarized HSI activation. (2) Comprehensive experiments have been conducted including synthetic and real experiments to demonstrate the superiority of BiSRNet. The ablation study is also extensive. The performance of BiSRNet is impressive, which surpasses existing SOTA BNNs by huge margins, over 2.5 dB.

Weaknesses

(1) Some details are missing. For example, how did you get the 3D mask $\mathbf{M}^*$? I remember the coded aperture used in CASSI is 2D. Its shape should be $H \times W$. Why does it have an additional dimension here? And how? (2) Some critical experimental analyses are lacking. For instance, at the end of section 4.4, “Binarizing Different Parts”, why binarizing the bottleneck can reduce the most parameters? And why binarizing the decoder can achieve the largest Ops reduction? An analysis should be provided to explain this. (3) I remember Binary Connect [39] binarizes the weights of CNN. But it seems that the activations of Binary Connect [39] are also binarized in Table 1. Why did you do that? Could you please explain this?

Questions

Please refer to Weaknesses

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

This paper proposes BiSRNet for efficient and practical HSI restoration from compressed measurement in snapshot compressive imaging (SCI) systems. But this paper has not discussed the limitations of the proposed method. I suggest the author discuss the potential and limitations of the proposed method in more detail, which will make the contributions of the proposed method more significant.

Summary

This work focuses on studying the binarized spectral compressive imaging reconstruction problem. A Binarized Spectral-Redistribution Network (BiSRNet) is proposed. The authors first design a basic U-Net as the base model to begin the binarization. Then a Binarized Spectral-Redistribution Convolutional (BiSR-Conv) unit is proposed to replace the 32-bit convolutional layer of the base model to derive BiSRNet. The BiSR-Conv has two advantages than vanilla 1-bit convolutional layer: (i) can redistribute the spectral distributions before binarization. (ii) has a scalable hyperbolic tangent function to approach the Sign function more closely.

Strengths

The novelty and motivation are good. First of all, the research topic is new. Nobody explores binarized spectral compressive imaging before. The authors not only contribute a good method but also conduct experiments using before BNNs designed for other topics. We should respect this. Secondly, the redesigned base model is well motivated. The architecture shows the insight of “full-precision information flow”. The idea of redistributing HSI representations before binarization is interesting and reasonable. The Tanh(\alpha x) is also amazing. Flexibly controlling the gap with Sign function is really cool. + The performance is good and solid. The proposed BiSRNet outperforms the SOTA BNNs widely used in image classification. Meanwhile, the computational and memory complexity of BiSRNet is much lower than full-precision CNNs while the performance is comparable. In the real HSI experiments, it seems that the proposed BiSCI performs better than CNN-based methods in noise suppression. + The experiments are comprehensive, not only quantitative but also qualitative comparisons shows the advantages of the proposed method. The ablation study is sufficient to verify the effectiveness of the proposed techniques. + The codes and pre-trained weights are provided in the supplementary. The reproducibility can be checked and ensured. + The writing is good and easy-to-follow. The figures, tables, and mathematical notations are very clear.

Weaknesses

- The definition is a little confusing. Sign(0) = 0 and Tanh(0) = 0. But in this paper, Sign(0) and Tanh(0) is defined as -1, as shown in Eq.(3) and line 152. Why? More explanation should be provided. - Although the proposed BNN achieves good performance with CNN. But compared with Transformer-based methods, there is a large gap. I understand that binarizing Transformer is not easy because of the self-attention mechanism. But it is good idea to keep trying.

Questions

Will you also plan to release the code of other BNNs compared in Table 1?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

See the above comments

Summary

In this paper, binarized neural network is first utilized in hyperspectral image reconstruction. For model binarization, authors propose the Binarized Spectral-Redistribution Convolution (BiSR-Conv), which adaptively redistributes the HSI representations before binarizing activation. Since the Sign function is non-differentiable, a scalable hyperbolic tangent function is applied in backpropagation, which has less approximation error. In BiSR-Conv, the additional residual connection encourages the full-precision information to propagate throughout the whole network. Based on BiSR-Conv, four binarized convolutional modules are designed to address the dimension mismatch issue during feature reshaping.

Strengths

An application to a new domain; Clear and consecutive writing; A feasible idea for deploying spectral compression imaging in edge devices.

Weaknesses

Experiments are insufficient. How does the ‘full-precision information’ affect the whole network? Is the ‘full-precision information’ play a dominant role in BiSR-Conv or not?

Questions

In Sign Approximation experiment, the proposed scalable hyperbolic tangent function and previous Sign approximation functions should be compared in other BNNs, to further verify the generalization of the proposed function.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

yes

Summary

This paper proposes a Binarized Neural Network based approach known as BiSRNet for binarized HSI restoration. The main motivation of the paper stems from the fact that any CNN or transformer-based architectures are computationally heavy for devices with low computing power and hence need extremely fast and light weight networks like binary neural networks. This paper redesigns the binary conv layer (BiSR-Conv) to exploit the distributed nature of HSI representations. They also employed a scalable Tanh function to decrease the approximation error. Overall, this paper is well-written and reasons well for various changes to the design of BCNNs.

Strengths

- Well-motivated problem and design of the solution - Good empirical evidence - Good presentation - Clear to understand

Weaknesses

- Main weakness of this paper is their redesigned conv modules are only applicable to the HSI domain? If so this limits the scope and impact of this work. - Lack of theory

Questions

- I am not very familiar with HSI work so I do not have many questions at this point.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Please see the weakness.

Thank you for addressing my comments

Thank you for addressing my comments, especially on 3D mask derivation. The paper offers many insights into binarized spectral compressive imaging. I'll keep my score of 7. Thank you.

Thanks for the response

The response from the authors has well addressed my concerns. I also read the comments from other reviewers. Now I am convinced that the work has both novelty and contributions to the fields of both compact NN design and spectral compressive imaging. Therefore, I would like to raise my score to Accept.