Speed Tests

Speed is the primary measure by which broadband connections are sold. We offer a variety of different mechanisms to measure broadband speed.

Supported Tests

TCP and WebSockets speed tests are available on all Device Agents including agents installed on routers and on phones running Android or iOS. All UDP speed tests are only available on Device Agents within routers.

TCP Speed Tests

TCP (Transmission Control Protocol) speed tests measure the download and upload speed of the broadband connection in bits per second. The transfer is conducted over one or more concurrent HTTP connections (using the GET method for download and the POST method for uploads).

In the download speed test, the agent fetches a portion of an infinitely sized binary (non-zero, randomly generated) payload hosted on an HTTP server on the target test server. The content is discarded as soon as it is received.

In the upload speed test, the agent generates the payload itself (using “/dev/random” as a non-blocking source of random content) to send to the server. The upload speed is measured at the server side (the receiver).

TCP Test Methodology

Both download and upload speed tests operate for a fixed duration by default. We recommend a duration of five seconds. Optionally, we can apply a transfer size limit (specified in MB). We recommend a fixed-duration test as it will cater well for all broadband access speeds. However, you may prefer a fixed-volume test to measure bandwidth usage. Speak to your account manager to make any of these adjustments to your speed tests.

We support four separate variants of the test:

  • Single TCP connection download speed test.

  • Multiple TCP connection download speed test.

  • Single TCP connection upload speed test.

  • Multiple TCP connection upload speed test.

For multiple TCP connection tests, we typically recommend using eight concurrent connections. In some cases (e.g., where the product of bandwidth and delay between agent and server is very high), it may be necessary to increase the number of concurrent connections.

The HTTP test can be configured to force the use of the ‘cubic’ TCP congestion control algorithm (which better represents real-world conditions) when testing against our HTTP server application, and this is our default.

Factors such as TCP slow-start are accounted for through the use of a “warm-up” period. This period begins as soon as the test starts and seeks to establish that the throughput has reached a stable rate before starting the real test (which will continue over the same TCP connection(s)). The warm-up period ends after a minimum period (default 1.5 seconds) when either a stable rate is achieved, or when a preconfigured time limit (default 5 seconds) is hit. It is important to note that we exclude the data transferred in the warm-up period from the main test results, but we still record it separately as a supplementary metric that we can make available to you on request.

TCP Test Metrics

The TCP speed test client captures the following key metrics:

  • The speed, in bytes per second.

  • The amount of data transferred.

  • The duration of the test.

  • Details about the warm-up period.

  • The hostname and IP address of the test server.

  • The number of TCP retransmissions that occurred during the test.

The agent may also record these values at multiple intervals during the test. This is commonly used to help characterize the difference between 'burst' and 'sustained' throughput (where transfer speeds may be inflated at the start of a TCP connection). Speed is measured at the application layer, so on a 100 Mbit/s physical link the maximum measurable speed is approximately 95 Mbit/s, due to packet overheads.

Finally, our TCP speed test makes use of hardware-specific functionality on some hardware devices. This is typically used on devices where the CPU is insufficiently powerful to saturate the WAN link in software mode alone. The nature of these implementations varies; sometimes it uses a dedicated hardware co-processor, and sometimes it uses a special kernel driver to avoid unnecessary copying data. An example of this is where we have integrated with Broadcom's "BSPEED" kernel driver in their DOCSIS chipsets and SPDT (single pole double throw) accelerators, as well as Airoha's TC3162 accelerator.

TCP Test Example

We noticed Fibre Max underperforming in our study Measuring Broadband New Zealand. After an investigation the cause was located and a fix rolled out. In the chart below, our TCP speed test shows a significant improvement in average download speed after 4 September for RSP X. Average speeds go from under 750Mbps to over 900Mbps which is in-line with what we would expect to see on Fibre Max lines. RSP Y made their changes on 19 October; a similar step change in performance can be seen following these changes.

This chart shows the average download speed from two anonymized ISPs, X and Y, which made changes to their network following our Fibre Max investigation in New Zealand.

UDP Speed Tests

The UDP (User Datagram Protocol) speed tests measure downstream and upstream throughput at the application layer using UDP.

This tests support both software- and hardware-accelerated modes of operation. In the software-accelerated mode, the traffic is generated and measured using the main CPU in the device, in userspace. In hardware-accelerated mode, the test makes use of platform-specific hardware acceleration features to generate or receive UDP packets at a far higher speed than the main CPU can support.

Hardware acceleration is currently supported on certain generations of Broadcom and Qualcomm SoCs (systems-on-chip). The measurement traffic is not directly handled by the main CPU, and instead uses a separate network co-processor. For Broadcom, measurements of up to 10 Gbps are attainable using the licensed 'speed service' API functionality from Broadcom. On Qualcomm, measurements of up to 10 Gbps are also attainable using their Nanalog Streaming Service (NSS) UDP APIs. We have performed multiple integrations with each of these implementations.

We recommend avoiding using hardware acceleration for speed tests, unless unavoidable, because these features may be subject to additional licensing fees (as with Broadcom) or may make troubleshooting more difficult.

UDP Test Methodology

Regardless of whether hardware acceleration is used or not, the measurement operates in the same way. For a download test, the agent issues a download test request to our test server over a TCP-based control channel, requesting that the server begin transmitting UDP test traffic back to the agent at a fixed rate. The agent then begins to receive the measurement packets, and assesses the throughput received. If the throughput is at least 80% of the requested rate, then the agent requests that the server increase its sending rate, and the process restarts. Eventually, the agent will stop requesting increases in transmission rates by the server, and the agent will measure the received rate at the current transmission rate. This process is controlled by a 'speed ladder', which we can configure for you.

For an upload test, the UDP speed test client issues a notification to the test server over TCP, informing it that transmission of UDP test traffic is about to commence. The agent then begins transmitting measurement packets. Packets are transmitted at full-line rate immediately and for the entire duration of the test. The packet size can be configured, and is 1400 bytes by default.

In both the upstream and downstream directions, the throughput is measured over a five-second period by default (excluding any ladder ramp-up time). The speed is always measured at the receiver side.

Finally, there is a ‘dual-channel’ extension to the hardware-accelerated mode of transmission. The dual-channel approach uses a software-based UDP measurement stream in parallel with the hardware-accelerated UDP stream. This mode is to cater specifically for certain legacy hardware that has the main CPU connected via a 1 Gbps internal interface, and the network co-processor connected via another 1 Gbps internal interface. Used individually, measurements could never exceed 1 Gbps, even on a >1 Gbps WAN link. However, when used in parallel, measurements over 1 Gbps can be achieved.

UDP Test Metrics

The following key metrics are captured by the UDP test for both downstream and upstream:

  • Measured throughput.

  • Transmission rate.

  • Transmission bytes.

  • Transmission duration.

  • Server hostname and IP.

  • Packet size.

Lightweight Capacity Speed Tests (UDP)

The lightweight capacity speed tests measure the instantaneous capacity of the link using a small number of UDP packets. The tests support both downstream and upstream measurements, conducted independently.

Lightweight Speed Test Methodology

In the downstream mode, the test agent handshakes with the test server over TCP, requesting a fixed number of packets to be transmitted back to the agent . The agent specifies the transmission rate, number of packets and packet size in this handshake. The agent records the arrival times of each of the resulting packets returned to it.

In the upstream mode, the agent again handshakes with the test server, this time informing it of the characteristics of the stream it is about to transmit. The agent then transmits the stream to the server, and the server locally records the arrival times of each packet. At the conclusion of this stream, the agent asks the server for its view of the arrival time of each packet.

With this resulting set of arrival times, the agent calculates the throughput achieved. This throughput may be divided into multiple windows, and an average taken across those, to smooth out buffering behavior.

This test uses up to approximately 99% less data than the conventional (TCP or UDP) speed tests and completes in a fraction of the time (100 milliseconds versus 10 seconds). Of course, there are trade-offs with such a vast reduction in data usage. The accuracy is lower than the regular TCP or UDP speed tests, and this varies by access technology. Generally, DSL or fiber connections produce results within a few percent of the TCP or UDP measurements. Results are more varied on DOCSIS connections, due to the increased variability of latency on this access technology.

Lightweight Test Metrics

The lightweight speed test agent captures the following key metrics:

  • Number of packets sent and number of packets received.

  • Packet size in bytes.

  • Total size received (in bytes).

  • Time to receive all packets.

  • Transfer speed.

  • Number of out-of-order packets.

WebSockets Speed Tests

The WebSockets speed tests measure the download and upload speed of the broadband connection in bits per second. The transfer is conducted over one or more concurrent TCP connections. WebSockets and HTTP (using the Fetch API) are used via HTML5 APIs as the transport for all measurement traffic. This requires the web browser to support HTML5.

In the download speed test the agent repeatedly fetches chunks of data from the target test server. The content is discarded as soon as it is received. The agent determines from the browser which is the best transport protocol to use; some browsers perform better with WebSockets and others perform better using the Fetch API.

In the upload test the agent generates the payload itself to send to the server. Similarly, the agent repeatedly uploads chunks of this data to the target test server, which immediately discards the content.

Our lab tests have demonstrated that figures of 10 Gbps are attainable on relatively common hardware (e.g., the 2020 Apple Mac M1) using modern web browsers.

WebSockets Test Methodology

The speed tests (both download and upload) operate for a fixed duration specified in seconds. We can impose a maximum limit on transfer volume instead if data volumes are of concern.

Both the download and upload tests use eight concurrent TCP connections by default. The size of the chunk to transfer will vary between 16 KB and 1 MB and will be determined automatically during the warm-up phase of the test.

Factors such as TCP slow-start are accounted for using a "warm-up" period. This period begins as soon as the test starts and seeks to remove the impact of TCP slow start from the results and serves to determine the number of TCP connections to be used for the remainder of the test. It is important to note that the data transferred in the warm-up period is excluded from the main test results.

WebSockets Test Metrics

The WebSockets speed test agent records:

  • Throughput.

  • Bytes transferred.

  • Test duration.

WebSockets Test Example

Cisco Real Speed uses our WebSockets speed test to compare in-home performance on a personal device to the service provided at the router.

PreviousNetworkNextLatency, Loss, Disconnections, and Jitter Tests

Last updated