Quantized Minimum Error Entropy Criterion

Comparing with traditional learning criteria, such as mean square error, the <italic>minimum error entropy</italic> (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyi’s entropy estimator, called <italic>information potential</italic> (IP), is a popular MEE cost in <italic>information theoretic learning</italic>. The computational complexity of IP is, however, quadratic in terms of sample number due to double summation. This creates the computational bottlenecks, especially for large-scale data sets. To address this problem, in this paper, we propose an efficient quantization approach to reduce the computational burden of IP, which decreases the complexity from <inline-formula> <tex-math notation="LaTeX">$O({N^{2}})$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$O({MN})$ </tex-math></inline-formula> with <inline-formula> <tex-math notation="LaTeX">$M \ll N$ </tex-math></inline-formula>. The new learning criterion is called the <italic>quantized MEE</italic> (QMEE). Some basic properties of QMEE are presented. Illustrative examples with linear-in-parameter models are provided to verify the excellent performance of QMEE.