Deep neural networks have achieved human-level accuracy on almost all\nperceptual benchmarks. It is interesting that these advances were made using\ntwo ideas that are decades old: (a) an artificial neuron based on a linear\nsummator and (b) SGD training.\n However, there are important metrics beyond accuracy: computational\nefficiency and stability against adversarial perturbations. In this paper, we\npropose two closely connected methods to improve these metrics on contour\nrecognition tasks: (a) a novel model of an artificial neuron, a "strong\nneuron," with low hardware requirements and inherent robustness against\nadversarial perturbations and (b) a novel constructive training algorithm that\ngenerates sparse networks with $O(1)$ connections per neuron.\n We demonstrate the feasibility of our approach through experiments on SVHN\nand GTSRB benchmarks. We achieved an impressive 10x-100x reduction in\noperations count (10x when compared with other sparsification approaches, 100x\nwhen compared with dense networks) and a substantial reduction in hardware\nrequirements (8-bit fixed-point math was used) with no reduction in model\naccuracy. Superior stability against adversarial perturbations (exceeding that\nof adversarial training) was achieved without any counteradversarial measures,\nrelying on the robustness of strong neurons alone. We also proved that\nconstituent blocks of our strong neuron are the only activation functions with\nperfect stability against adversarial attacks.\n