CAdViSE or how to find the sweet spots of ABR systems

With the recent surge in Internet multimedia traffic, the enhancement and improvement of media players, specifically Dynamic Adaptive Streaming over HTTP (DASH) media players happened at an incredible rate. DASH Media players take advantage of adapting a media stream to the network fluctuations by continuously monitoring the network and making decisions in near real-time. The performance of algorithms that are in charge of making such decisions was often difficult to be evaluated and objectively assessed from an End-to-end or holistic perspective [1].

CAdViSE provides a Cloud-based Adaptive Video Streaming Evaluation framework for the automated testing of adaptive media players [4]. We will introduce the CAdViSE framework, its application, and propose the benefits and advantages that it can bring to every web-based media player development pipeline. To demonstrate the power of CAdViSE in evaluating Adaptive Bitrate (ABR) algorithms we will exhibit its capabilities when combined with objective Quality of Experience (QoE) models. Our team at Bitmovin Inc. and ATHENA laboratory has selected the ITU-T P.1203 (mode 1) quality evaluation model in order to assess the experiments and calculate the Mean Opinion Score (MOS), and better understand the behavior of a set of well-known ABR algorithms in a real-life setting [2]. We will display how we tested and deployed our framework using a modular architecture into a cloud infrastructure. This method yields a massive growth to the number of concurrent experiments and the number of media players that can be evaluated and compared at the same time, thus enabling maximum potential scalability. In our team's most recent experiments, we used Amazon Web Services (AWS) for demonstration purposes. Another awesome feature of CAdViSE that will be discussed here is the ability to shape the test network with endless network profiles. To do so, we used a fluctuation network profile and a real LTE network trace based on the recorded internet usage of a bicycle commuter in Belgium.

CAdViSE produces comprehensive logs for each experimental session. These logs can then be applied against different goals, such as objective evaluation or to stitch back media segments and conduct subjective evaluations. In addition, startup delays, stall events, and other media streaming defects can be imitated exactly as they happened during the experimental streaming sessions [3].