Hierarchical Uncertainty Exploration via Feedforward Posterior Trees

Summary

The paper proposes a method to obtain a hierarchical representation of samples from the posterior distribution of inverse problems. This method can be integrated into any existing approach to learn and sample from the posterior distribution. It involves learning a tree structure where each node represents a prototype reconstruction, with child nodes refining their parent node. This allows for the exploration of different modes and levels of detail in the solution space.

Strengths

**Originality.** While the method has connections to post-hoc hierarchical clustering of samples, it is novel in that it allows for faster visualization of the posterior distribution samples at test time. **Quality and clarity.** Overall, the paper is well-structured, easy to follow, and sufficiently motivates the problem. However, the clarity of the presentation of the proposed loss function could be improved. **Significance.** The problem of informatively visualizing the variability in samples from the posterior distribution of inverse problems is impactful.

Weaknesses

* While I truly appreciate the importance of informatively visualizing samples from the posterior distribution, the proposed method seems to require several heuristics for training to yield desirable results (e.g., discussions in sections 3.3 and 3.4). Are the computational complexity gains of the proposed method, compared to performing post-hoc hierarchical clustering, worth the need for tuning the parameters in these heuristics? * It seems like the architecture of the network required to predict the tree depends on the tree depth and width as $K^d$, where $d$ is the tree depth and $K$ is the branching factor. This appears to be a very limiting factor in the depth and width of the tree that can be learned. How does the cost of learning the tree structure as we scale $K$ and $d$ compare to the cost of post-hoc hierarchical clustering?

Questions

* When augmenting existing methods for learning conditional generative models (for learning the posterior), how does the loss associated with the tree structure affect the learning of the generative model? How do the authors prevent the proposed loss from negatively biasing the learning of the posterior distribution? * What is the computational overhead of the proposed method compared to simply training a generative model to learn the posterior distribution? What is the cost of post-hoc hierarchical clustering? How many inverse problems need to be solved to justify the extra training time?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Authors provide a discussion on the limitations of their method.

Summary

This work proposes a technique to predict a tree-structured hierarchical summarization of a posterior distribution using a single forward pass of a neural network. The technique is an amortized hierarchical version of the oracle loss in multiple choice learning. Experiments show the method is effective at hierarchically representing the posterior both qualitatively and quantitatively.

Strengths

1. The method is simple, efficient, and sound. 2. It addresses a fundamental and practically relevant problem (hierarchically visualizing complex distributions). 3. The presentation of the method and the experiments is excellent.

Weaknesses

1. Table 1 reports number of function evaluations, but it's not clear how that translate to runtime since the time per function call and extent of parallelization varies . 2. The qualitative evaluation in Figure 1 and Figure 4 are not very informative. It's often hard to tell the differences between many of the nodes or judge whether there is an unambiguous underlying hierarchical structure. 3. The clustering / hierarchy produced the method is based on $L_2$ distance, which is not a very useful metric in many applications including images. Though as the authors discussed, this can be potentially addressed e.g. by first embedding the inputs with an autoencoder. Empirically demonstrating that this limitation can be addressed would be important for adoption of this method in many applications.

Questions

1. Can you apply this method when the reconstruction loss is not an $L_2$ loss but a generic function? 2. How does the runtimes compared in Table 1? 3. Can you show how the methods compare as a function of runtime in Table 1? It's possible that you don't need to run that many function evals for the baselines to get similar performance.

Rating

7

Confidence

3

Soundness

4

Presentation

4

Contribution

4

Limitations

The authors discussed limitations of the paper.

Summary

This work proposes to solve the problem of quantifying and visualising the uncertainty in the solutions of ill-posed inverse problems like image-to-image translation, image re-construction, inpainting, etc. The paper proposes to do so by using 'Posterior-trees' where the authors make use of the result that optimizers of Eq. (1) form CVT of the posterior. Further application of simple Bayes rule and simple probability allows for construction of a hierarchal tree to quantify as well as visualize the uncertainty associated with the predicted solution of the inverse problem at hand.

Strengths

1. The paper is well written with clear ideas and rationale. 2. The idea of using hierarchal trees to quantify and visualize the uncertainty associated with the posterior of inverse problem is indeed novel. Although the core idea on which the paper builds upon is already presented in [45]. 3. Nonetheless, extending the results of [45] to construct a heirarchal version of it is notable. 4. The results and visualizations are satisfying.

Weaknesses

1. I think the authors should provide more elaborate background on CVT and results of [45], either in main text or in supplementary for ease of the reader. 2. One thing that I did not understand is - what is the trade-off between breadth and depth of the constructed tree? For example - what is the difference between having more children nodes with low depth and having less children nodes with high depth. This would help in further understanding the advantages and limitations of the proposed method. 3. Another point that I am a bit skeptical about is that the method operates directly in the image space (or the space of the input of the problem at hand). In case of images, this space is very high-dimensional. In such high-dimensional spaces, capturing all the variations of the solution specially through a discrete tree-like structure is problematic. I don't know how the proposed method is able to handle it. Overall I enjoyed reading the paper, and the paper certainly seems to be novel, albeit, incrementally. Hence, I lean for weak acceptance.

Questions

See weaknesses

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

See weaknesses

Response to Rebuttal

Thank you for your response. I am convinced with the response to my questions, further the response provided by the authors to Reviewer VHHJ helped me in further clarification. I am increasing my score accordingly!