Analysis of Video Feature Learning in Two-Stream CNNs on the Example of Zebrafish Swim Bout Classification

Semmelhack et al. (2014) have achieved high classification accuracy in\ndistinguishing swim bouts of zebrafish using a Support Vector Machine (SVM).\nConvolutional Neural Networks (CNNs) have reached superior performance in\nvarious image recognition tasks over SVMs, but these powerful networks remain a\nblack box. Reaching better transparency helps to build trust in their\nclassifications and makes learned features interpretable to experts. Using a\nrecently developed technique called Deep Taylor Decomposition, we generated\nheatmaps to highlight input regions of high relevance for predictions. We find\nthat our CNN makes predictions by analyzing the steadiness of the tail's trunk,\nwhich markedly differs from the manually extracted features used by Semmelhack\net al. (2014). We further uncovered that the network paid attention to\nexperimental artifacts. Removing these artifacts ensured the validity of\npredictions. After correction, our best CNN beats the SVM by 6.12%, achieving a\nclassification accuracy of 96.32%. Our work thus demonstrates the utility of AI\nexplainability for CNNs.\n