Configuring Connected Devices Tests
For some tests (for example, all of the dynamic application test suites), you can deploy the tests with a few clicks using just the Global Settings screen. For others, you must configure the global settings and set a target for each test. While these steps are the minimum requirement to deploy tests, you can also customize each test according to your needs. This article explains how to configure the required global and individual settings, as well as how to configure other common test fields and where to find information about more advanced settings.
Global Settings Configuration
Whether you’re creating a suite of tests via a template (see Configuring Connected Devices Templates), or some of your own via Create Custom Tests, you must first configure your global settings which apply to all of the tests you’re creating (except for some tests’ interval settings - see #3, below). Once you configure your global settings, you can then customize any individual tests. Note that some templates have some global settings automatically configured. To configure the global settings:
If creating a custom test, click + Add Test on the left to select one or more tests from the list.
Add Test list If using a template, the tests are already pre-selected.
Decide Which agents should we run tests from?: Choose from All agents or Specific agents.
If you choose Specific agents, another field appears where you can type in individual agents.
Set the Interval: The frequency with which your tests run. Choose from 15 minutes up to 24 hours.
When setting up custom tests, the global interval setting only applies to the following tests: Download speed, Upload speed, HTTP server, Video streaming, Gameplay, Social media, and Game store. The Continuous UDP stream test automatically runs approximately every 1.5 seconds. All other tests default to an 8-hour interval. To change the interval settings for these latter tests, you must do so in their individual test settings screens.
Test templates have different default interval configurations, specified in Configuring Connected Devices Templates.
You can also choose to create a “Testing Window” via the + Add Window button, which is a daily recurring time slot when tests run, such as 6 pm to 10 pm. You can create one or multiple testing windows (click + Add Row), each with configurable intervals. Intervals have the same range as above, as well as “once per window”. Note: windows cannot overlap, nor can an interval be larger than the window.
To remove the window and return to the interval field, click the “-“ (minus) sign to the right of the window setup.
Testing window
Name this test group: Give your test(s) a name if creating custom tests. Templates have their template name added by default, though you may edit it. See Test Groups for more information about test groups.
Tests Requiring Target Configuration
For all custom tests that aren’t dynamic application test suites, you also need to specify a target for each test before you can deploy them. Any test that requires a target has a red warning circle next to it. All templates except web experience have targets set by default.
Click on each test.
Select or type in the target.
Target field on an individual test If you do not require any further customization, click Review Template, then Deploy Now.
Configuring Common Test Settings
Once you configure your global settings, you can customize each individual test via a range of common configuration options, plus some fields that are unique to each test.
Changes you make to one test apply to that test only. If you make no other changes, the global settings still apply to all the other tests. If you change a field that is not part of the global settings, such as Protocol type, the global settings still apply to that test. You only override the global setting for an individual test if you change the same setting on the test as on the global setting, such as interval or selected agents.
Common Test Configuration Fields
Every test has four common elements within the test’s Basic Settings tab. Every test also has an Advanced Settings tab, where you can further tweak the test’s settings. We explain below what the common elements are and how to configure them.
Test Name: You must give your test a unique name.
If unaltered, a custom test name defaults to the name of the test, such as “HTTP server”. Names of tests within templates default to include the template, test, and often the target.
If a default test name already exists, ascending numbers are appended to additional tests of the same type, for example, “HTTP server (1)”, “HTTP server (2)”.
When you create the test group name, the test name is prepended with the test group name. For example an HTTP server test with the test group name “All Users” becomes “All Users – HTTP server”. See Test Groups for more information.
Target: As detailed above, you must set the endpoint(s) for custom tests – whether a server, a streaming service, or a CDN, for example. Some tests have dropdown options, some you must type in, while our test suites come preconfigured. For all but web experience templates, targets are set by default.
Many custom tests require you to choose whether you select a single target, select from a target set, manually type in a target, or route via the first external hop. Not all tests offer all four options.
Choose a single target from a dropdown list of available targets, and filter by On-/Off-net and Public/Private targets.
Single target dropdown options Target sets are groups of single targets that automatically route your test to the nearest target or one selected at random. Note: target sets do not send a test to multiple targets, but are sets within which a single target is chosen automatically based on your choice of nearest vs. random. Target sets can additionally be filtered by Public/Private.
Target set dropdown options Manual input allows you to type in an IP address or domain name of your choosing.
Manual target input field Choose first external hop (for ping tests only) when you want to measure round trip latency to the first publicly routable hop instead of specifying a target. This is often the CMTS (cable modem termination system) in a cable network or the OLT (optical line terminal) in a fibre network.
First external hop selection
Interval: Override the global or default interval value using this setting. See #3 in Global Settings Configuration for more information.
Which agents should we send the data from?: Override the global value using this setting. See #2 in Global Settings Configuration for more information.
Test Groups
Test Group names are a kind of tag that can help you keep track of which tests were created for what purpose. Whether you’re creating several of the same template with different configurations, or your own custom groups of tests, you’ll need to make each group distinct from another. Give your groups unique names that meaningfully represent the outcomes of their measurements, such as “Marketing Campaign” for the Network Tests template or “London Speed Tests” for a custom group targeting speed tests in London only.
Adding Test Group Names
Once you name your group, the group name gets prepended to each test name within the group. For example, if your “London Speed Tests” group includes download and upload speed tests, as well as the Burst UDP stream test, each test will show up on your Tests screen as “London Speed Tests - Download speed”, “London Speed Tests - Upload speed”, and “London Speed Tests - Burst UDP stream”.
You may have created other groups for other cities, too, such as Birmingham and Manchester. So, on your Tests screen, you may encounter three seemingly identical Download speed tests, but you know which one is which by its Test Group tag: London Speed Tests - Download speed, Birmingham Speed Tests - Download speed, and Manchester Speed Tests - Download speed.
Once you start typing in the name field, your test group name will mirror as a tag beneath the field. You can continue to edit the test group name before moving to the next step, and the tag automatically updates accordingly.
Other Configuration Options
Every test is unique, so every test usually has a number of additional fields within the basic and advanced settings tabs that you can configure. Find a link to articles about each test within Connected Devices Tests, wherein you’ll find information about each test’s specific configurable fields.
Slider Configurations
Some configuration options in the platform use a slider to adjust a range of values. Each slider uses set intervals to move from value to value along the slider. For example, a slider with an interval of 100 ms would move from 3,000 ms to a higher value in jumps of 100 ms starting at 3,100 to 3,200 to 3,300, etc.
It's important to note that when creating tests using the API, intervals are not specified for the same fields that use sliders on the platform's UI. This is on purpose to allow more complex setups via the API. Therefore, a value can be set via the API that is unsupported via the slider on the UI. For example, you may set the value of the field to 3,333 ms via the API, but this does not conform to an interval of 100 ms. In instances where you configure the test using the API, then view it on the UI, you will encounter the following behavior.
The range slider on the UI shows the value you set via the API (e.g., 3,333).
If you do not interact with the slider, i.e. you are just viewing the test in the UI, and you then leave the test screen, the value stays as 3,333.
If you click on or move the slider, the slider jumps to the nearest permitted interval (e.g., 3,300), and on submit this new value (3,300) is saved.
Bandwidth Estimations
When you go to review your template, a column shows an estimated monthly bandwidth for a 1 GB line, based on the configurations for the tests you’ve set. If you change the configuration of your tests, the bandwidth estimation changes in accordance. We provide some examples below.
Test
Estimated bytes per test execution
Expected data usage per execution on a 1 Gbit line (1 Gbit being 940 Mbit/s as the wireline speed of a 1 Gbit ethernet port over TCP)
Example (For 1 agent, running every hour, monthly) (except Continuous UDP)
Continuous UDP stream (Disconnections, Latency, Loss)
9.3Bps
9.3 bytes * (30 days divided by 1.5 seconds) = 16.1 MB
Download / Upload speed (TCP)
Variable according to line speed
The variation depends on how quickly the line gets up to stable speed, e.g., for a 1 Gbit line an average per 30 day month running once per day = between (812,500,000 * 1 * 30) and (1,250,000,000 * 1 * 30) = 24.37 - 37.5 GB/month/agent Minimum data usage per execution on a 1 Gbit line in bytes: ((“Minimum warm up time” + “Measurement duration”) * 125,000 bytes)) * 940 e.g. ((1.5 + 5) * 125,000) * 940 = 763,750,000 bytes = 763.8 MB Maximum data usage per execution on a 1 Gbit line in bytes: ((“Maximum warm up time” + “Measurement duration”) * 125,000 bytes) * 940 e.g. ((5 + 5) * 125,000) * 940 = 1,175,000,000 bytes = 1.175 GB
For 1 Gbp line, min warmup: 1.5 s, measurement duration: 5 s 1 * 24 * 30 * 763.8 MB = 549.9 GB (min) For 1 Gbp line, max warmup: 5 s, measurement duration: 5 s 1 * 24 * 30 * 1.175 GB = 846 GB (max)
Netflix (Video Streaming Test)
Expected data use depends on result achieved and line speed. The amount of data downloaded depends on streaming speed. from the Netflix OCA cache, as it is effectively a single-threaded download.
2.35 GB
1 * 24 * 30 * 2.35 GB = 1.7 TB
Last updated