Testing Thresholds
What Testing Thresholds Are
There are times when we hold back on scheduled testing because a device used for testing (often a router) is running at or over capacity on some metric. We call these testing thresholds. Testing at these times would impact the user’s experience of their device. So, we stop any scheduled test in preference for a time when testing wouldn’t overburden the device.
Threshold checks apply to scheduled tests only; they do not apply to instant tests. Instant tests override thresholds and scheduled tests to run as a priority, after which scheduled testing resumes.
Types of Testing Threshold
We currently employ two types of scheduled testing threshold. The first is CPU (central processing unit) usage, or CPU load, and the second is called cross-traffic.
CPU Load
If a test is scheduled to run and the agent detects the CPU load of the testing device is above a configurable threshold value, then the test will not run. The default threshold value for CPU load is 30%.
Cross-Traffic
Cross-traffic is the term we use to refer to network activity on a user's broadband connection that is not generated by us. The cross-traffic threshold check is powered by our data usage test, which monitors the volume of traffic passing through the device hosting the Device Agent.
It is important that scheduled tests do not run in the presence of cross-traffic (or anything more than a very minimal amount of cross-traffic). The reasons for this are two-fold:
Cross-traffic would impact our measurement results, meaning that we would not be accurately characterizing the user's broadband performance.
Our measurement traffic may adversely impact the application that is generating the cross-traffic (e.g., video streaming or online gaming).
Our router and Android agents come with cross-traffic detection. The default acceptable cross-traffic threshold is 25,000 bytes per second (250 kbps) combined downstream and upstream. These limits can be remotely configured as part of the test schedule assigned to the Device Agent. The cross-traffic detection strategies vary according to the Device Agent type in use.
How the Thresholds Work
If CPU overload or cross-traffic is detected when a scheduled test is due to be executed, the Device Agent will pause and wait for a 5-second window where CPU and cross-traffic levels are below the thresholds. If the levels are below the thresholds, the test will open a 30-second window in which to execute test measurements, during which as many scheduled tests as possible are executed. If 30 seconds pass with no opportunity to test, the queued tests are skipped. The levels of CPU usage and cross-traffic, along with the currently configured threshold and other details of the interface it was detected on, are recorded and submitted to our data pipeline.
To find out more about how else we minimize cross-traffic occurrences, see Scheduled Tests.
The chart below illustrates cross traffic on a home network and under which circumstances a scheduled test would or would not run.
Last updated