BBC iPlayer Streaming Tests

The BBC iPlayer streaming test is an application-specific test, supporting the streaming of video and audio content from iPlayer using their protocols and codecs. The iPlayer test is geo-restricted to the UK only.

iPlayer Test Methodology

The test begins by fetching a list of the most popular videos from an iPlayer XML API. The most popular video is chosen for playback, on the basis that this is most representative of what users will be watching at the time. Moreover, if there are iPlayer caches present in the ISP's (internet service provider's) network then this content is more likely to be cached there.

The XML manifest for this video is then fetched from the BBC's web servers. This contains paths to all of the different bitrates that this video is encoded at, and the different CDNs (content delivery networks) that serve them. BBC iPlayer uses multiple CDNs to serve content. By having the probe directly fetch the XML manifest, we can ensure that any decisions the BBC make (e.g. “ISP X should always be served by CDN Y”) are followed by our test. The test parses the XML manifest file, honoring the priority assigned to each CDN and building an ordered list of available bitrates.

At this point the test can begin to fetch video content from the content server. This is currently achieved over RTMP (Real-Time Messaging Protocol), mimicking the iPlayer web browser client. The test parses video frames as it goes, capturing the timestamp contained within each frame. After each frame, we sample how much real-time has elapsed versus video time. If the video timestamp is greater than the amount of real time elapsed in a sample period, then an underrun has not occurred. Otherwise, one has occurred.

Key Metric Measured

As with the YouTube test, the agent is configured to start testing at the highest supportable bitrate and then step down if and when stalls occur. This allows us to identify the 'bitrate reliably streamed'.

The key outputs from this metric are:

• The bitrate reliably streamed.

• The start-up delay (the time taken to download two seconds of video).

• The TCP connection time.

• The number of stalls and their duration (this is only applicable if the test is not running in the 'bitrate reliably streamed' mode).

• The downstream throughput achieved.