Utilizing Explainable AI for Quantization and Pruning of Deep Neural Networks

For many applications, utilizing DNNs (Deep Neural Networks) requires their\nimplementation on a target architecture in an optimized manner concerning\nenergy consumption, memory requirement, throughput, etc. DNN compression is\nused to reduce the memory footprint and complexity of a DNN before its\ndeployment on hardware. Recent efforts to understand and explain AI (Artificial\nIntelligence) methods have led to a new research area, termed as explainable\nAI. Explainable AI methods allow us to understand better the inner working of\nDNNs, such as the importance of different neurons and features. The concepts\nfrom explainable AI provide an opportunity to improve DNN compression methods\nsuch as quantization and pruning in several ways that have not been\nsufficiently explored so far. In this paper, we utilize explainable AI methods:\nmainly DeepLIFT method. We use these methods for (1) pruning of DNNs; this\nincludes structured and unstructured pruning of \\ac{CNN} filters pruning as\nwell as pruning weights of fully connected layers, (2) non-uniform quantization\nof DNN weights using clustering algorithm; this is also referred to as Weight\nSharing, and (3) integer-based mixed-precision quantization; this is where each\nlayer of a DNN may use a different number of integer bits. We use typical image\nclassification datasets with common deep learning image classification models\nfor evaluation. In all these three cases, we demonstrate significant\nimprovements as well as new insights and opportunities from the use of\nexplainable AI in DNN compression.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC