As edge devices become prevalent, deploying Deep Neural Networks (DNN) on\nedge devices has become a critical issue. However, DNN requires a high\ncomputational resource which is rarely available for edge devices. To handle\nthis, we propose a novel model compression method for the devices with limited\ncomputational resources, called PQK consisting of pruning, quantization, and\nknowledge distillation (KD) processes. Unlike traditional pruning and KD, PQK\nmakes use of unimportant weights pruned in the pruning process to make a\nteacher network for training a better student network without pre-training the\nteacher model. PQK has two phases. Phase 1 exploits iterative pruning and\nquantization-aware training to make a lightweight and power-efficient model. In\nphase 2, we make a teacher network by adding unimportant weights unused in\nphase 1 to a pruned network. By using this teacher network, we train the pruned\nnetwork as a student network. In doing so, we do not need a pre-trained teacher\nnetwork for the KD framework because the teacher and the student networks\ncoexist within the same network. We apply our method to the recognition model\nand verify the effectiveness of PQK on keyword spotting (KWS) and image\nrecognition.\n