DEANN: Speeding up Kernel-Density Estimation using Approximate Nearest Neighbor Search

Kernel Density Estimation (KDE) is a nonparametric method for estimating the\nshape of a density function, given a set of samples from the distribution.\nRecently, locality-sensitive hashing, originally proposed as a tool for nearest\nneighbor search, has been shown to enable fast KDE data structures. However,\nthese approaches do not take advantage of the many other advances that have\nbeen made in algorithms for nearest neighbor algorithms. We present an\nalgorithm called Density Estimation from Approximate Nearest Neighbors (DEANN)\nwhere we apply Approximate Nearest Neighbor (ANN) algorithms as a black box\nsubroutine to compute an unbiased KDE. The idea is to find points that have a\nlarge contribution to the KDE using ANN, compute their contribution exactly,\nand approximate the remainder with Random Sampling (RS). We present a\ntheoretical argument that supports the idea that an ANN subroutine can speed up\nthe evaluation. Furthermore, we provide a C++ implementation with a Python\ninterface that can make use of an arbitrary ANN implementation as a subroutine\nfor kernel density estimation. We show empirically that our implementation\noutperforms state of the art implementations in all high dimensional datasets\nwe considered, and matches the performance of RS in cases where the ANN yield\nno gains in performance.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC