With the emergence of smart cities, Internet of Things (IoT) devices as well\nas deep learning technologies have witnessed an increasing adoption. To support\nthe requirements of such paradigm in terms of memory and computation, joint and\nreal-time deep co-inference framework with IoT synergy was introduced. However,\nthe distribution of Deep Neural Networks (DNN) has drawn attention to the\nprivacy protection of sensitive data. In this context, various threats have\nbeen presented, including black-box attacks, where a malicious participant can\naccurately recover an arbitrary input fed into his device. In this paper, we\nintroduce a methodology aiming to secure the sensitive data through re-thinking\nthe distribution strategy, without adding any computation overhead. First, we\nexamine the characteristics of the model structure that make it susceptible to\nprivacy threats. We found that the more we divide the model feature maps into a\nhigh number of devices, the better we hide proprieties of the original image.\nWe formulate such a methodology, namely DistPrivacy, as an optimization\nproblem, where we establish a trade-off between the latency of co-inference,\nthe privacy level of the data, and the limited-resources of IoT participants.\nDue to the NP-hardness of the problem, we introduce an online heuristic that\nsupports heterogeneous IoT devices as well as multiple DNNs and datasets,\nmaking the pervasive system a general-purpose platform for privacy-aware and\nlow decision-latency applications.\n