Summary
This paper presents a Laplace approximation-based probabilistic retrieval approach (aka. Bayesian metric learning for image retrieval). The author provides a probabilistic view of the contrastive loss based on the von-Mises Fisher distribution and corrections for the Hessian positive definiteness. Extensive experimental evaluations are performed, demonstrating the advantages of the presented approach in calibrated uncertainties, out-of-distribution detection, close-set and open-set retrieval, and ablations on a few parameters of design choices.
Strengths
[Originality] This paper presents the use of Laplace approximation instead of amortized inference for probabilistic retrieval. Although similar ideas have been proposed to improve the amortization gap of variational autoencoders in several prior works, its development in the context of metric learning is novel. This includes the introduction of a probabilistic view of the contrastive loss and the utilization of the Hessian approximation based on GCN.
On the other hand, previous Bayesian metric learning approaches have also been proposed, such as hedge instance embedding (HIE) and probabilistic face embeddings (PFE). In this regard, the novelty of this work lies in the contrastive learning-based loss and the learning of stochastic embeddings based on the Laplacian autoencoder (Miani, M. et al., 2022).
Miani, M., Warburg, F., Moreno-Muñoz, P., Skafte, N., & Hauberg, S. (2022). Laplacian autoencoders for learning stochastic representations. *Advances in Neural Information Processing Systems*, *35*, 21059-21072.
[Quality] Extensive empirical evaluations, including a careful ablation study, are conducted. The results demonstrate that the proposed approach outperforms HIE and PFE in the considered cases.
[Clarity] The paper effectively uses figures to demonstrate ideas. However, I do find the need of reading with jumps between the main text and the supplemental material in order to grasp the concept. The organization can be improved.
[Significance] This paper demonstrates uncertainty quantification based on Laplace approximation in the context of probabilistic retrieval. This is an important topic for improving the robustness and mitigating the silent failure of deep neural network systems. The main contribution of this paper appears to be the empirical study, which can be informative and useful for practitioners.
Weaknesses
It appears that this paper builds upon prior work, including the Laplacian autoencoder (Miani, M. et al., 2022), as well as several works on uncertainty in metric learning. Consequently, I am more concerned about the unique technical contributions of this work. In this regard, the necessity of the probabilistic view of the contrastive loss is not well presented and not well motivated.
Another missing piece is how to perform test-time retrieval within the proposed framework and how its efficiency compares to the amortized approach. Given a query image, does the proposed approach learn a stochastic representation for it? Is the retrieval based on ranking the deterministic similarities between the query image and all other candidate images? If not, how is it performed?
The organization and clarity of this paper could be improved. If certain results are deemed important enough, they should be moved into the main text. The authors should focus on introducing the unique aspects of this work and provide a strong motivation for them, such as the Von Mises-Fisher distribution. The frequent references to the Appendix disrupt the logical flow of the paper.
Questions
The probabilistic view in the supplemental also seems a little bit weird to me since both argument and parameter are data-dependent. Would it be sufficient to just treat the probabilistic contrastive likelihood as a second-order differentiable loss, if they are mathematically equivalent anyway?
What are the main roadblocks that have been overcome in applying the Laplacian autoencoder (LAE) to probabilistic retrieval?
Are there any other ways of approximating Hessian for contrastive loss, while still maintains scalability and positive definiteness? Is KFAC applicable, e.g., Ritter et al., 2018?
Ritter, H., Botev, A., & Barber, D. (2018, January). A scalable laplace approximation for neural networks. In 6th International Conference on Learning Representations, ICLR 2018-Conference Track Proceedings (Vol. 6). International Conference on Representation Learning.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
The authors discussed limitations about computation load which I think is reasonable.