JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services

Deep learning models are being deployed in many mobile intelligent\napplications. End-side services, such as intelligent personal assistants,\nautonomous cars, and smart home services often employ either simple local\nmodels on the mobile or complex remote models on the cloud. However, recent\nstudies have shown that partitioning the DNN computations between the mobile\nand cloud can increase the latency and energy efficiencies. In this paper, we\npropose an efficient, adaptive, and practical engine, JointDNN, for\ncollaborative computation between a mobile device and cloud for DNNs in both\ninference and training phase. JointDNN not only provides an energy and\nperformance efficient method of querying DNNs for the mobile side but also\nbenefits the cloud server by reducing the amount of its workload and\ncommunications compared to the cloud-only approach. Given the DNN architecture,\nwe investigate the efficiency of processing some layers on the mobile device\nand some layers on the cloud server. We provide optimization formulations at\nlayer granularity for forward- and backward-propagations in DNNs, which can\nadapt to mobile battery limitations and cloud server load constraints and\nquality of service. JointDNN achieves up to 18 and 32 times reductions on the\nlatency and mobile energy consumption of querying DNNs compared to the\nstatus-quo approaches, respectively.\n