Bitrate Adaptation and Guidance With Meta Reinforcement Learning

Abstract

Adaptive bitrate (ABR) schemes enable streaming clients to adapt to time-varying network/device conditions for a stall-free viewing experience. Most ABR schemes use manually tuned heuristics or learning-based methods. Heuristics are easy to implement but do not always perform well, whereas learning-based methods generally perform well but are difficult to deploy on low-resource devices. To make the most out of both worlds, we earlier developed <monospace>Ahaggar</monospace>, a learning-based scheme executing on the server side that provides quality-aware bitrate guidance to streaming clients running their own heuristics. <monospace>Ahaggar</monospace>'s novelty is the meta reinforcement learning approach taking network conditions, clients’ statuses and device resolutions, and streamed content as input features to perform bitrate guidance. <monospace>Ahaggar</monospace> uses the new Common Media Client/Server Data (CMCD/SD) protocols to exchange the necessary metadata between the servers and clients. While <monospace>Ahaggar</monospace> was a significant step forward, in this study, we focus on three open areas, namely, (i) exploring the performance of <monospace>Ahaggar</monospace> in a heterogeneous environment including both <monospace>Ahaggar</monospace> and non-<monospace>Ahaggar</monospace> clients with varied network conditions and device resolutions, and (ii) quantifying the impact of device resolutions on QoE with <monospace>Ahaggar</monospace>. We thoroughly investigate these areas and report our findings. We also (iii) discuss the <monospace>Ahaggar</monospace> design choices. Experiments on an open-source system show that <monospace>Ahaggar</monospace> adapts to unseen conditions fast and outperforms its competitors in several viewer experience metrics.