We appreciate the additional comments from Reviewer zt3G. Please note that these issues have also been raised by other reviewers, and we have addressed them in the previous responses. Regarding novelty, please refer to our response to reviewer boCg, and for issues related to the novelty of (and motivation for) hierarchical encoding, we have clarified addressed this in our response to reviewer JKXU.
For convenience, we summarize our replies and provide some additional information here.
**The overall novelty**
1\. We utilized parameter efficient layers for INR-based video compression. In INR-based compression, the number of parameters is directly related to the compression performance. However, all previously published work has employed layers with high complexity, e.g. sub-pixel convolution. In contrast, in HiNeRV, we use a bilinear interpolation layer, depth-wise convolutional layers and MLPs as the network layers. Despite the simplicity of this innovation, we have clearly shown that significant gains can be obtained by adopting parameter efficient layers.
2\. We also proposed hierarchical encoding, which encodes the relative positional information. Hierarchical encoding is parameter efficient as it scales with the upsampling ratio of the layer, rather than the full spatial dimensions. By adding multiple hierarchical encodings at different layers, we can represent high resolution signals, without the need for feature grids with high storage cost. When fixing the number of parameters, hierarchical encoding provided significant improvement for highly dynamic video sequences where previous works (e.g. NeRV/HNeRV) performed poorly (0.78/1.06 dB in PSNR for the Jockey/ReadySetGo sequence).
In our reply to reviewer JKXU, we mentioned that the high performance of HiNeRV with the MLP block is due to the use of the hierarchical encoding. To validate this, we performed an additional experiment and found that if we remove the hierarchical encoding from HiNeRV (with MLP blocks) then the average PSNR decreased by 2.54 dB. HiNeRV with MLP blocks, each of which only has a very lightweight convolutional layer in the stem, clearly outperformed the state-of-the-art CNN-based approach, HNeRV. The high performance of the MLP-based network is also different from the observation in previous work (the NeRV paper), which reported that MLP-based networks perform poorly in the context of the video encoding task. This further confirms the effectiveness of hierarchical encoding. We believe the use of the hierarchical encoding could also benefit MLP-based INR, but this remains for future research.
3\. While existing INR approaches for video compression are either frame- or patch-based, we introduced a unified representation. Our proposed HiNeRV, as a unified representation, can thus be utilized with either frames or overlapped patches. It yields better performance than a frame-wise representation, but also benefits from the parallelism of a patch-wise representation which can reduce the minimum memory requirement for computation. The performance improvement is confirmed in the ablation study (Section 4.3 in the paper), where we observe that HiNeRV with the unified representation obtained up to 0.49 dB PSNR improvement over the frame-wise configuration, on the UVG dataset.
We also found that the improvement is more significant when the encoding time is reduced. We performed an additional experiment following the settings used in Section 4.1 with only 37 epochs, and we observed that HiNeRV with the unified representation obtained 0.93 dB improvement in average PSNR compared with the frame-wise configuration. Moreover, with our unified representation, we can perform computation using patches without sacrificing performance - an important characteristic for INR-based video compression, due to the higher signal dimensionality. For example, a recently published paper [1] had to divide the videos into patches due to memory constraints.
**Writing of the paper**
We appreciate the comments provided by the reviewers regarding the paper's writing. We will revise the paper in order to enhance its clarity and readability.
[1] He et.al, Towards Scalable Neural Representation for Diverse Videos, CVPR 2023