Mutual Information Neural Estimation with Eta-Trick

A computing device receives a data X and Y, each having N samples. A function f(x,y) is defined to be a trainable neural network based on the data X and the data Y. A permuted version of the data Y is created. A loss mean is computed based on the trainable neural network f(x,y), the permuted version of the sample data Y, and a trainable scalar variable η. A loss with respect to the scalar variable η and the trainable neural network is minimized. Upon determining that the loss is at or below the predetermined threshold, estimating a mutual information (MI) between a test data XT and YT. If the estimated MI is above a predetermined threshold, the test data XT and YT is deemed to be dependent. Otherwise, it is deemed to be independent.