Advances in HTTP Adaptive Streaming: New Standards, Video Codecs, and...

The applications of video streaming are the primary drivers of Internet traffic, as over 82% of IP traffic in 2022. The prevailing technology to stream live and video-on-demand (VoD) content over the Internet is HTTP Adaptive Streaming (HAS). In HAS, the content is encoded at multiple representations (bitrate-resolution pairs), and delivered incrementally using segments (or chunks) of encoded representations. This allows for dynamic selection of different representations during playback, and the HAS model supports delivery under different and changing network- and device-specific conditions. As the demand for video streaming applications is on the rise, improved streaming methods, as well as video codecs and video content optimization algorithms, are being developed to meet this demand. This tutorial first presents a detailed overview of the existing file formats, protocols (HLS, DASH, CMAF), video codecs (H.264, HEVC, AV1, VVC), and quality assessment methods (PSNR, SSIM, VMAF, P.1203, CTA2066) used for streaming. Particular emphasis will be given to new video codecs, such as Versatile Video Coding (VVC), and their features applicable to streaming. We will then focus on recent advances in video encoding optimizations for HAS streaming. We will then introduce per-title, content-aware, and context-aware encoding methods, which optimize bitrate ladders based on the characteristics of videos and other contexts. It will also be presented how representations are selected in a way that bitrate ladders are optimized over various dimensions, including spatial resolution, frame rate, energy consumption, device type, and network condition. Additionally, methods for reducing the latency of dynamic ladder generation and fast multi-rates encoding will be covered. Finally, we will discuss various areas of recent progress in the design of streaming systems, including end-to-end modelling, optimization, and analysis, and highlight remaining open problems in the field.