Game Store Test Suite

Modern console games are delivered from online stores such as Steam, XBox Live and Playstation Network. The games can be tens or even hundreds of gigabytes, so achieving high throughput from these stores is important to gamers.

The throughput will depend on the CDNs (content delivery networks) that host the games. Internet service providers (ISPs) will have varying connection agreements with CDNs, which means that your performance could vary significantly depending on the ISP you use and where the CDNs are located.

Supported Platforms

Our game store test suite supports the following platforms:

• Steam

• XBox Live

• Playstation Network

Game Store Test Methodology

In all cases, our test downloads a portion of a real game for a fixed duration (20 seconds by default), or when a maximum transfer size has been reached (50 Mb by default), whichever happens first. The download is performed over HTTP, using multiple parallel requests. The platforms vary in their parallelization strategies, but typically use between 4 and 20 parallel TCP connections. Our test defaults to using 8 parallel connections, but this is configurable.

Factors such as TCP slow-start are accounted for through the use of a “warm-up” period. This period begins as soon as the test starts and seeks to remove the impact of TCP slow-start from the results. The warm-up period ends after a configurable amount of time (three seconds by default), or when a maximum warm-up size has been reached (1 Mb by default), whichever happens first.

Note: The data transferred in the warm-up period is excluded from the main test results, but it is still recorded separately as a supplementary metric.

Optionally, a “randomize data” function is available which can reduce the impact of caching on recorded speeds.

Game Store Test Metrics

The test captures the following key metrics:

• Download speed.

• A list of the CDN hostnames and IPs that were used in the download.

• Average DNS lookup time.

• Average TCP connection establishment time.

• Total size and duration of the transfer.