Summary
The paper introduces a novel framework called Self-Distilled Depth Refinement (SDDR) to enhance depth refinement, which aims to infer high-resolution depth maps with fine-grained edges from low-resolution depth estimations. The authors propose modeling depth refinement as a noisy Poisson fusion problem, addressing local inconsistency and edge deformation noises. The SDDR framework consists of depth edge representation and edge-based guidance. Through coarse-to-fine self-distillation, SDDR generates low-noise depth edge representations, which serve as pseudo-labels to guide the refinement process. The method demonstrates significant improvements in accuracy, edge quality, efficiency, and generalizability across five different benchmarks.
Strengths
1. Innovative Approach: The modeling of depth refinement as a noisy Poisson fusion problem is a novel and insightful approach that effectively addresses common issues in depth refinement.
2. Robust Framework: The self-distillation technique employed in SDDR enhances robustness against noise, resulting in high-quality depth maps with accurate edges.
3. Comprehensive Evaluation: The authors conduct extensive experiments across five benchmarks, showcasing the method's superior performance in various scenarios.
4. Efficiency: SDDR achieves higher efficiency compared to two-stage tile-based methods, reducing computational costs while maintaining or improving accuracy and edge quality.
5. Generalizability: The framework demonstrates strong generalizability, performing well on both synthetic and real-world datasets.
Weaknesses
1. Real-World Application: The paper primarily focuses on benchmarks and does not provide extensive discussion on real-world applications and potential limitations in practical scenarios.
2. Edge Case Handling: The method’s performance in handling extreme edge cases or highly noisy data is not thoroughly explored.
3. Ablation Studies: More detailed ablation studies are needed to understand the contribution of each component within the SDDR framework.
Questions
1. In Line 107-108, the logic between the previous works and your motivation is unsmooth; it would be better to provide more analysis.
2. In Figure 2, the author didn't show the edge deformation visualization. What kind of deformation would occur in depth estimation? Most off-the-shelf depth estimators have not produced deformed results in depth edges or other regions.
3. Do the results in state-of-the-art work like DepthAnything-V2 exhibit the artifacts, consistency, and deformations proposed by the authors?
4. Can you provide more insights into the performance differences between synthetic and real-world datasets?
5. The performances compared to other works seem not very significant due to the combination of several contributions in this paper. How can the authors judge whether the gains are from randomness of training, the settings of the models' parameters, or other factors?
6. What is the training time comparison between SDDR and other state-of-the-art methods?
7. How sensitive is the SDDR framework to hyperparameter settings?
8. How does SDDR perform on extreme edge cases with very high noise levels?
9. Can SDDR be adapted for real-time applications, and what modifications would be necessary?
Limitations
The paper provides extensive quantitative and qualitative results showcasing the strengths of the SDDR framework. However, it lacks a detailed analysis of potential failure cases or scenarios where the method does not perform well, which could provide insights for further improvements.