A Note on the Kullback-Leibler Divergence for the von Mises-Fisher distribution

We present a derivation of the Kullback Leibler (KL)-Divergence (also known as Relative Entropy) for the von Mises Fisher (VMF) Distribution in d dimensions.

stat.ML Mathematics Computer Science

View graph Source

Paper

References (8)

01Clustering on the Unit Hypersphere using von Mises-Fisher DistributionsA. Banerjee, I. Dhillon, Joydeep Ghosh et al.2005 · Journal of machine learning research · 1.1k citations

02Tutorial on Practical Prediction Theory for ClassificationJohn Langford2005 · Journal of machine learning research · 374 citations

03Algorithm AS 86: The Von Mises Distribution FunctionK. Mardia, P. Zemroch1975 · 53 citations

04Statistics of Directional DataK. Mardia1972 · 3.6k citations

05Dispersion on a sphereR. Fisher1953 · Proceedings of the Royal Society of London. Series A, Mathematical and physical sciences · 5.9k citations

06Handbook Of Mathematical FunctionsT. Gerber2016 · 6.7k citations

07Statistical inference of Langevin distribution for directional dataYoko Watamori1996 · 14 citations

08Bayesian inference for the von Mises-Fisher distributionK. Mardia, S. El-atoum1976 · 104 citations

Similar papers

von Mises-Fisher distributions and their statistical divergenceToru Kitagawa, Jeff Rowley2022 · arXiv (Cornell University) In Library

On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian DistributionsYufeng Zhang, Wanwei Liu, Zhenbang Chen et al.2021 · Neural Information Processing Systems · 80 citations In Library

Variational Approach for Efficient KL Divergence Estimation in Dirichlet Mixture ModelsSamyajoy Pal, Christian Heumann2024 · arXiv (Cornell University) In Library

Limit Distribution Theory for KL divergence and Applications to Auditing Differential PrivacyS. Sreekumar, Z. Goldfeld, Kengo Kato2023 · International Symposium on Information Theory · 5 citations In Library

Optimal Bounds between $f$-Divergences and Integral Probability MetricsR. Agrawal, Thibaut Horel2020 · International Conference on Machine Learning · 50 citations In Library

KL Divergence Between Gaussians: A Step-by-Step Derivation for the Variational Autoencoder ObjectiveAndrés M. Muñoz, Rodrigo Ramele2026 · arXiv.org · 0 citations In Library

Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergenceClémentine Chazal, Anna Korba, Francis Bach2024 · arXiv (Cornell University) In Library

An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient FlowCarles Domingo-Enrich, Aram-Alexandre Pooladian2023 · arXiv (Cornell University) In Library

Taming numerical imprecision by adapting the KL divergence to negative probabilitiesSimon Pfahler, Peter Georg, Rudolf Schill et al.2023 · arXiv (Cornell University) In Library

Tighter Variational Representations of f-Divergences via Restriction to Probability MeasuresAvraham Ruderman, Mark D. Reid, Dario García-García et al.2012 · International Conference on Machine Learning · 69 citations In Library

Patents near this paper

VARIATIONAL RELEVANCE VECTOR MACHINEUS 6,879,944 · MICROSOFT CORPORATION · 2005 DISTANCE MEASURE FOR PROBABILITY DISTRIBUTION FUNCTION OF MIXTURE TYPEUS 6,993,452 · AT&T CORP. · 2006 ACCELERATED DISCRETE DISTRIBUTION CLUSTERING UNDER WASSERSTEIN DISTANCEUS 10,013,477 · THE PENN STATE RESEARCH FOUNDATION · 2018 KL-DIVERGENCE KERNEL REGRESSION FOR NON-GAUSSIAN FINGERPRINT BASED LOCALIZATIONUS 8,463,291 · ALCATEL-LUCENT USA INC. · 2013 DISPARITY MITIGATION IN MACHINE LEARNING-BASED PREDICTIONS FOR DISTINCT CLASSES OF DATA USING DERIVED INDISCERNIBILITY CONSTRAINTS DURING NEURAL NETWORK TRAININGUS 11,676,037 · FAIRNESS-AS-A-SERVICE, INC · 2023

Source of record

arXiv