Generic Streaming (HLS/DASH) Tests

Video streaming services have largely standardized around HLS (HTTP Live Streaming) and DASH (Dynamic Adaptive Streaming over HTTP) as the two most common streaming methods. These are used by major streaming providers, and also by ISPs offering their own video streaming services.

Our HLS/DASH test supports measurement of any standards-compliant HLS/DASH stream. This includes both on-demand and live streaming content. The only input to the test is the path to a manifest file (usually a URL) that defines the video stream and its available encodings. This is one differentiator from the video streaming test suite, in that you choose the manifest file while the other video streaming tests test to predefined files. Additionally, you must choose the CDN provider (e.g., Akamai, Limelight, or custom), highlighted in the manifest.

HLS/DASH Test Methodology

The test can operate in two modes:

• fixed-bitrate mode - whereby it identifies how frequently stalls occur and for how long, or

• an adaptive mode - whereby it seeks to find the bitrate that can be reliably streamed without stalls

The HLS/DASH test also supports reporting VMAF (Video Multimethod Assessment Fusion) scores. VMAF is a perceptual scoring mechanism developed by Netflix. It works by calculating reference scores against the different encodings of the source video, under optimal conditions, and then comparing the received video stream against these.

Key Metrics Measured

This test reports the following key metrics:

• The bitrate reliably streamed.

• The startup delay (the time taken to download two seconds of video).

• The TCP connection time.

• The number of stalls and their duration (this is only applicable if the test is not running in the 'bitrate reliably streamed' mode).

• The downstream throughput achieved.

• The VMAF score.