In this paper, we propose a novel defensive transformation that enables us to\nmaintain a high classification accuracy under the use of both clean images and\nadversarial examples for adversarially robust defense. The proposed\ntransformation is a block-wise preprocessing technique with a secret key to\ninput images. We developed three algorithms to realize the proposed\ntransformation: Pixel Shuffling, Bit Flipping, and FFX Encryption. Experiments\nwere carried out on the CIFAR-10 and ImageNet datasets by using both black-box\nand white-box attacks with various metrics including adaptive ones. The results\nshow that the proposed defense achieves high accuracy close to that of using\nclean images even under adaptive attacks for the first time. In the best-case\nscenario, a model trained by using images transformed by FFX Encryption (block\nsize of 4) yielded an accuracy of 92.30% on clean images and 91.48% under PGD\nattack with a noise distance of 8/255, which is close to the non-robust\naccuracy (95.45%) for the CIFAR-10 dataset, and it yielded an accuracy of\n72.18% on clean images and 71.43% under the same attack, which is also close to\nthe standard accuracy (73.70%) for the ImageNet dataset. Overall, all three\nproposed algorithms are demonstrated to outperform state-of-the-art defenses\nincluding adversarial training whether or not a model is under attack.\n