KD-Lib: A PyTorch library for Knowledge Distillation, Pruning and Quantization

In recent years, the growing size of neural networks has led to a vast amount\nof research concerning compression techniques to mitigate the drawbacks of such\nlarge sizes. Most of these research works can be categorized into three broad\nfamilies : Knowledge Distillation, Pruning, and Quantization. While there has\nbeen steady research in this domain, adoption and commercial usage of the\nproposed techniques has not quite progressed at the rate. We present KD-Lib, an\nopen-source PyTorch based library, which contains state-of-the-art modular\nimplementations of algorithms from the three families on top of multiple\nabstraction layers. KD-Lib is model and algorithm-agnostic, with extended\nsupport for hyperparameter tuning using Optuna and Tensorboard for logging and\nmonitoring. The library can be found at - https://github.com/SforAiDl/KD_Lib.\n