YouTube Streaming Tests
The YouTube streaming test is an application-specific test, supporting the streaming of video and audio content from YouTube using their protocols and codecs.
YouTube Test Methodology
The test begins by selecting a representative video that supports the full range of video resolutions up to UHD quality (2160p).
The video identifier is downloaded from one of our servers. In order to retrieve a manifest of available quality levels, a specific sequence of messages are exchanged with the server. First, the API key is identified by downloading the web page for the video. Then, the YouTube API is invoked using HTTP POST. This gives access to media information required by the test. Details of how to perform these tasks may change over time (e.g., key extraction or which HTTP header fields to use). It is controlled by our server-side configuration parameters that are downloaded when the test starts.
By making this request from the probe we ensure that the test is receiving from the same content server as the user would if they were using a desktop computer on the same connection.
The test then connects to the content server (using whatever server YouTube would normally direct a real client on the same connection to) and begins streaming the video and audio.
The test parses video frames as it goes, capturing the timestamp contained within each video frame. After each frame, we sample how much real-time has elapsed versus video time. If video time is greater than real-time in a sample period, then an underrun has not occurred. Otherwise, one has occurred.
The test downloads 10 seconds of audio and video at a time, with a buffer of 40 seconds. So, on start-up, the test immediately downloads (at full speed) 40 seconds of video and audio, and then downloads more as required, keeping the 40 second playback buffer full. By default, the test runs for a fixed duration of 20 seconds of real-time.
Key Metric Measured
In its default mode of operation, the test captures the 'bitrate that can be reliably streamed' on the user's connection. This is achieved through the following process:
Find the fastest recent speedtest result that the probe has completed.
As described above, fetch the list of YouTube videos, find the most popular one, then select the highest bitrate encoding that is less than the fastest speedtest result found in step 1.
Attempt to stream this video, for a fixed duration of 20 seconds of real-time.
If successful, then the “bitrate reliably streamed” for this instance is the bitrate that we just fetched.
However, if a stall event occurs, then we immediately abort the test and retry at the next lower bitrate.
If we find a bitrate that we can stream without a stall event occurring, then that bitrate is our “bitrate reliably streamed” for this instance.
However, if we encounter stalls for every bitrate, then the “bitrate reliably streamed” is zero.
The key outputs from this metric are:
The bitrate reliably streamed.
The startup delay (the time taken to download two seconds of video).
The TCP connection time.
The number of stalls and their duration (this is only applicable if the test is not running in the 'bitrate reliably streamed' mode).
Supported Codecs
MPEG4
WebM
Dash (adaptive)
Although the adaptive codec is supported, the test does not actually adapt its rate; we stream at full rate all the time, which provides for reproducibility.
Flash
Last updated