Recursive Offloading for LLM Serving in Multi-Tier Networks

Heterogeneous device-edge-cloud computing infrastructures have become the backbone of modern telecommunication operators and Wide Area Networks (WANs), providing multi-tier computational support for emerging intelligent applications. With the rapid proliferation of Large Language Model (LLM) services, efficiently coordinating inference tasks and reducing communication burden within these multi-tier network architectures becomes a critical deployment challenge. Current LLM serving paradigms exhibit significant limitations: on-device deployment restricts service to lightweight LLMs due to hardware constraints, while cloud-centric deployment encounters resource congestion and considerable prompt communication overhead during peak periods. Model-cascading inference, though better suited for multi-tier networks, depends on static, manually-tuned thresholds that cannot adapt to dynamic network conditions or varying task complexities. To address these challenges, we propose RecServe, a recursive offloading framework tailored for LLM serving in multi-tier networks. RecServe introduces a task-specific hierarchical confidence evaluation mechanism that guides offloading decisions based on inferred task complexity in progressively scaled LLMs across device, edge, and cloud tiers. To further enable intelligent task routing across tiers, RecServe employs a sliding-window-based dynamic offloading strategy with quantile interpolation, enabling real-time tracking of historical confidence distributions and adaptive offloading threshold adjustments. This design allows inference tasks to be recursively offloaded to higher tiers only when necessary, optimizing heterogeneous resource utilization while reducing cross-tier communication with little compromise on service quality. Theoretical analysis provides distinct conditions under which RecServe is expected to achieve reduced communication burden and computational costs. Experiments on eight datasets demonstrate that RecServe outperforms CasServe in both service quality and communication efficiency, and reduces the communication burden by over 50% compared to centralized cloud-based serving.