DCDepth: Progressive Monocular Depth Estimation in Discrete Cosine Domain

Summary

The paper presents a novel framework for the long-standing monocular depth estimation. The task is first formulated as a progressive regression in the discrete cosine domain. The authors propose two modules: the PPH module progressively estimates higher-frequency coefficients based on previous predictions, and the PFF module incorporates a DCT-based downsampling technique to mitigate information loss and ensure effective integration of multi-scale features.

Strengths

1. The paper is well-organized and the idea is easy to understand. 2. The results of the method are presented clearly.

Weaknesses

1. The authors claim that the global-to-local (coarse-to-fine) depth estimation is a contribution, but this idea is common and adopted by other works [1] [2]. \ [1] Liu C, Yang G, Zuo W, et al. Dpdformer: a coarse-to-fine model for monocular depth estimation[J]. ACM Transactions on Multimedia Computing, Communications and Applications, 2024, 20(5): 1-21.\ [2] Li Y, Luo F, Xiao C. Self-supervised coarse-to-fine monocular depth estimation using a lightweight attention module[J]. Computational Visual Media, 2022, 8(4): 631-647. 2. In Section 3.2, some descriptions are confusing. “This grouping strategy ensures that lower-frequency groups contain fewer components necessitating more prediction steps, while higher-frequency groups encompass a larger number of components requiring fewer steps”, but in Fig 2, local areas (higher-frequency) require more steps. It is recommended to explain the unclear representation. 3. The authors claim that the DCT-based downsampling technique tends to mitigate information loss, but this module has not been reasonably explained. 4. The experiments are somewhat lacking in terms of including the latest works that achieve state-of-the-art performance. Although the authors compare their results with some previous works, other MDE methods [3, 4, 5] could provide valuable additional comparisons. When evaluated against these newer methods, the proposed method does not demonstrate superior performance. \ [3] Ning J, Li C, Zhang Z, et al. All in tokens: Unifying output space of visual tasks via soft token[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 19900-19910. \ [4] Yang L, Kang B, Huang Z, et al. Depth anything: Unleashing the power of large-scale unlabeled data[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 10371-10381. \ [5] Saxena S, Kar A, Norouzi M, et al. Monocular depth estimation using diffusion models[J]. arXiv preprint arXiv:2302.14816, 2023. 5. “MDE is extensively applied across various fields such as autonomous driving, robotics, and 3D modeling [45, 48, 9, 42]” recommend changing to “autonomous driving[…], robotics[…], and 3D modeling[…]”.

Questions

See the Weaknesses.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

The contributions are not substantial enough. The authors have not clearly articulated their design for DCT-based downsampling. Additionally, the proposed method does not achieve state-of-the-art results.

Summary

The paper introduces a frequency domain-based method for monocular depth estimation. The proposed method begins with the prediction of low-frequency components to establish a global scene context, followed by successive refinement of local details through the prediction of higher-frequency components. The proposed method is validated on the NYU-Depth-V2, TOFDC and KITTI datasets.

Strengths

To the best of my knowledge, the discrete cosine domain-based progressive design is significantly different from previous methods and gives a promising paradigm in depth estimation. The proposed method is interpretable, using discrete cosine transformation to segregate the depth information into various frequency components and enriching details in a progressive manner. The proposed method outperforms prior work across multiple datasets with comparable or fewer parameters. Detailed ablation study indicate the effectiveness of each key component. The paper is clearly structured and well presented.

Weaknesses

As shown in Table 2, the proposed method significantly outperforms NewCRFs, PixelFormer, and IEBins on the TOFDC dataset compared to other datasets. It would be beneficial to provide a detailed discussion regarding this to better understand the strengths of the proposed method. It is nice to also visualize the evolution of depth predictions in the frequency domain. In Fig. 3, the three depth predictions at the bottom look the same. They probably need to be replaced with the actual experimental results.

Questions

Please see the weakness section.

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

It is nice to try the completed ground-truth depth maps for supervision on the KITTI dataset, using existing depth completion frameworks.

Summary

This paper introduces DCDepth, a framework designed to tackle the long-standing monocular depth estimation task. In general, the proposed methods are quite interesting, the motivations for this work are clear, the experiments conducted are comprehensive, and the paper is well-structured.

Strengths

This paper introduces DCDepth, a framework designed to tackle the long-standing monocular depth estimation task. In general, the proposed methods are quite interesting, the motivations for this work are clear, the experiments conducted are comprehensive, and the paper is well-structured.

Weaknesses

1. The comparisons with SoTA methods in Table 1 and Table 2 are not very clear. The authors should consider focusing on more decimal places and only highlighting the best-performing methods for each metric. 2. The tables should include the publication year and the venue of the compared methods, as this will help readers better understand the context of the comparisons.

Questions

Please see weaknesses.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A

Summary

The author has proposed the DC depth, which aims to predict a depth map from a monocular image. The authors introduce a novel technique that implements depth estimation of the frequency coefficients from the discrete cosine domain and enables modeling the local depth correlations. The author conducted experiments on the NYU-Depth-V2, ToF-DC, and KITTI datasets.

Strengths

1. Predicting depth from the frequency domain is a relatively novel insight. 2. The presentation is clear. Figure 1 clearly illustrates the progressive estimation scheme.

Weaknesses

Lack of comparison: Why not compare with methods like Depth Anything [Yang et al., CVPR 2024], Metric3D[Yin et al., ICCV 2023], etc.? The methods currently compared are outdated. Therefore, it cannot be said that a new state-of-the-art performance has been achieved. L14

Questions

Please consider answering the questions in weaknesses section during the rebuttal. Besides, on L27-28, the author states that the current depth estimation is based on a per-pixel basis. I am not sure if this statement is reasonable, as recent MDE methods, such as Depth Anything, they are based on transformers, and there are some operations of patch tokenization, perhaps the per-pixel basis is not a reasonable way of writing.

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

2

Limitations

yes.

Feedback after rebuttal

Dear reviewers, the authors put a significant effort into providing detailed answers to the questions raised in your evaluation reports. In particular, they discussed concerns about novelty and comparison to SOTA, which are very relevant. Please participate in the discussion by sharing your comments about the rebuttal. Thanks AC

Feedback to authors

Dear Reviewer HHm4 and fZWS, You still need to share your feedabck about the rebuttal; please post your comments as soon as possible. Thank you