Deep neural networks (DNN) are black box algorithms. They are trained using a gradient descent back propagation technique which trains weights in each layer for the sole goal of minimizing training error. Hence, the resulting weights cannot be directly explained. Using Topological Data Analysis (TDA) we can get an insight on how the neural network is thinking, specifically by analyzing the activation values of validation images as they pass through each layer.
Paper
References (5)
03Using Topological Data Analysis to Understand the Behavior of Convolutional Neural Networks www
04Cornell
05Human Performance Wing Airman Systems Directorate