Agent-to-Agent Test Overview
The agent-to-agent network test allows you to have ThousandEyes agents at both ends of a monitored path, so you can test the path in either or both of two directions: source to target or target to source. The agent-to-agent test can have multiple source agents monitoring the target agent. Similarly, the target agent monitors each of the source agents. By monitoring in both directions, asymmetrical paths between source and target agents can be visualized, and data for both paths is available.
The agent-to-agent test produces standard network metrics: packet loss, latency, jitter, and optionally throughput—an improved form of the bandwidth metric—along with Path Visualization and path MTU. Metrics are produced for either or both directions, depending on test configuration. You can configure the agent-to-agent test to use either TCP or UDP. You can specify the payload size in bytes, or the test can determine it automatically based on path MTU discovery.
How ThousandEyes Agent-to-Agent Tests Work
Agent-to-agent tests improve upon agent-to-server tests by basing metrics on packets traveling in only one direction, rather than needing to send probe packets and obtain responses. Rather than sending response packets, the receiving ThousandEyes agent provides the results metrics directly to the ThousandEyes collector, which a non-ThousandEyes receiver cannot do. To enable one-way metrics that require timing, the sending and receiving agents perform an exchange to determine the clock offset before sending the actual test packets. With the clock offset, packets can be timed in one direction.
When calculating end-to-end packet losses, this model results in a true one-way loss, rather than loss being possible in either direction. Metrics for trip time in an agent-to-agent test are one-way timings, rather than round-trip timings. To synchronize time between the sending and receiving agents, the sending agent transmits clock synchronization packets at the beginning of each test round, prior to the actual probe packets.
Clock Synchronization vs. Packet Loss: Clock synchronization is required in order to report accurate one-way latency and jitter. Packet loss for the probe stream is still derived from packets the target agent receives, so you can see loss results even when clock synchronization fails or is unstable. In that situation, latency and jitter might show no data while loss remains valid. If you see this pattern, inspect middleboxes (for example, stateful firewalls) that might interfere with the clock-sync preamble; see Network Tests, Explained for measurement details.
Similarly, the throughput metric in agent-to-agent tests measures the one-way data transfer rate between the endpoints. Having ThousandEyes agents at both ends allows a more accurate measurement compared to the bandwidth metric of an agent-to-server test, since the target agent is measuring incoming byte rate directly, rather than the sending agent inferring data rate from the target's responses to the incoming bytes.
You can configure the throughput test duration for between 5 seconds (default is 10 seconds) up to 30 seconds. You can achieve greater accuracy with longer test duration. TCP-based throughput testing uses payloads sent at the fastest possible rate. UDP-based throughput testing allows a rate setting, which can simulate Quality of Service (QoS) bandwidth shaping policies, for example.
For performing path visualization and measuring forwarding loss, the mechanism is not changed from the agent-to-server test.
In the preceding image, the upper paths travel from the agent on the left to the agent on the right, while the lower paths travel in the opposite direction. The direction is indicated by the arrows connecting the nodes on the paths. Using the Direction selector, you can display only the source to target, target to source, or both directions.
Understanding Agent-to-Agent Metrics
An agent-to-agent test provides metrics for each configured direction: Source to Target, Target to Source, and (for bi-directional tests) Both Directions.
Loss
Source to Target / Target to Source: One-way packet loss for that direction. Calculated by the receiving agent and reported to the platform.
Both Directions: Combined packet loss over the full round-trip journey. Calculated as a combined probability from two independent one-way packet streams.
Formula
Example: The source agent sends packets to the target agent, experiencing 20% one-way loss (an 80% success probability). In an independent stream, the target agent sends packets to the source agent, experiencing 10% one-way loss (a 90% success probability). The probability of a packet successfully traversing the full path is 0.8 * 0.9 = 0.72 (72% chance of success). The combined Both Directions loss is therefore 28% (100% - 72%).
The Both Directions loss value is not a simple average of the two one-way loss percentages. It measures the combined probability that a packet drops at any point during its full path. See the Alert Metrics Reference for more information.
Latency and Jitter
Source to Target / Target to Source: One-way latency or jitter for that direction.
Both Directions Latency: The sum of the Source-to-Target and Target-to-Source latency measurements.
Both Directions Jitter: The average of the Source-to-Target and Target-to-Source jitter measurements.
Agent-to-Agent Test Targets
When selecting a target agent and one or more source agents, there are three possible configurations of the agent-to-agent test:
Enterprise → Cloud: The source is an Enterprise Agent and the target is a Cloud Agent. The IP address used for the target is the public IP address of the Cloud Agent (all Cloud Agents have publicly routable IP addresses). The Cloud Agent's IP address will be available in the Target IP column on the Table tab of the Overview view, and displayed in a tooltip when hovering over the Cloud Agent, in the Path Visualization view.
Enterprise → Enterprise: The source is an Enterprise Agent and the target is an Enterprise Agent. The IP address used for the target is the value of the target for tests IP Address setting, on the Advanced Settings tab of the Agent Settings. The target Enterprise Agent's IP address will be available in the Target IP column on the Table tab of the Overview view, and displayed in a tooltip when hovering over the Enterprise Agent, in the Path Visualization view. If the target agent will be accessed through a NAT IP address that is automatically discovered by checking the Behind a NAT box in the Agent Settings, then the NAT IP address will be displayed in the tooltip with the target agent's IP address.
Cloud → Enterprise: The source is a Cloud Agent and the target is an Enterprise Agent. The IP address used for the target is the value of the target for tests IP Address setting, on the Advanced Settings tab of the Agent Settings. The target Enterprise Agent's IP address will be available in the Target IP column on the Table tab of the Overview view, and displayed in a tooltip when hovering over the Enterprise Agent, in the Path Visualization view. If the target agent will be accessed through a NAT IP address that is automatically discovered by checking the Behind a NAT box in the Agent Settings, then the NAT IP address will be displayed in the tooltip with the target agent's IP address.
A test can have both Cloud and Enterprise Agents as source agents.
For targeted agents behind a NAT, see NAT Traversal for Agent-to-Agent Tests. In general, for Enterprise Agents in the agent-to-agent test, check your firewall rules, NAT, or port forwarding configuration to ensure that test packets are permitted and that any needed IP address and port translations are performed.
Agent-to-Agent Test Settings
The Basic Configuration tab for an agent-to-agent test is shown in the following image.
Test Name: Name of the test.
Target Agent: Select the target agent for the test.
Interval: Configure the test interval expressed in minutes.
Agents: Select the source agents to monitor the target agent.
Direction: Select the direction of the monitoring as Source to Target, Target to Source, or Both Directions.
Protocol: Select either the TCP or UDP protocol.
Enable Throughput: Check this box to perform throughput measurements in the selected direction(s).
With TCP: Configure the duration of the throughput measurement, along with Simultaneous Connections.
Simultaneous Connections: Set the number of parallel TCP connections (1-32) used for throughput measurement. Multiple connections can help achieve higher total throughput by utilizing additional TCP streams and compute resources. A single connection can typically reach approximately 5 Gbps (Gigabits per second), depending on the processor speed. Therefore, values higher than 1 should only be used when needed.
With UDP: Configure the duration of the throughput measurement, along with Maximum Rate or Specified Rate that the agent will send packets at.
WARNING: Enable Throughput with UDP and Maximum Rate settings can be disruptive. The UDP protocol has no congestion control, therefore a UDP agent-to-agent test performed at Maximum Rate might saturate your agent's local connection for a set duration. If unsure, use TCP.
Path Trace Mode: When the In Session checkbox is selected, perform a path trace "in session" within an agent-to-agent test, by initiating a TCP session with the target server, and sending path trace packets within that session. NOTE: This option is available to accommodate customer environments where middleboxes like Palo Alto and Juniper firewalls interfere with the way the ThousandEyes platform collects path trace metrics. These products misinterpret the path tracing as malicious probes, so by first establishing the TCP session the firewall doesn't reject path trace packets.
Alerts: Check the Enable box to assign alert rules and create suppression windows.
The Advanced Settings tab for an agent-to-agent test is shown in the following image.
Server Port: Port number on the target agent to which a source agent sends packets.
MSS: Set the Maximum Segment Size to Auto or Manual, specifying the number in bytes to avoid fragmentation (the value can be between 30-1400 bytes).
Collect BGP Data: Check this box to enable the BGP Route Visualization view, and select which BGP monitors to use.
Transmission Rate: Check this box to choose the rate at which packets for measurements are sent (between 10 and 100 packets per second).
No. of Path Traces: By default, each agent uses three path trace packets to discover each hop in the Path Visualization view. Uncheck this box to change the default setting. You can set up to 10 packets.
DSCP: Configure the DSCP value for the packets.
Troubleshooting
Connection to Agent Failed / Clock Sync Errors
Before measuring performance, agents execute a clock synchronization task to ensure accurate latency measurements via a brief packet exchange.
If this initial exchange fails, a Connection to source/target agent failed error appears, and the platform collects no end-to-end metrics. This occurs when a stateful firewall or security device drops the clock sync packets.
Check for dropped packets on firewalls between the agents. Configuring the test to use the UDP protocol instead of TCP might resolve this issue.
Inconsistent Loss in Path Visualization vs. End-to-End Metrics
End-to-end metrics (loss, latency, jitter) and path visualization use separate measurement techniques. End-to-end metrics rely on a stream of 50 packets sent directly to the target. Path visualization builds paths using a traceroute-style probing method.
Because these features use different packets and methodologies, loss might appear in one view but not the other. Rely on end-to-end metrics as the primary performance indicator, and use path visualization as a diagnostic tool to investigate the path.
Additional Notes
Server Port: Setting the Server Port only specifies the target TCP or UDP port to be used for incoming packets to the target. Port value 22 is not allowed; in addition, the port must be greater or equal to 1024 when Cloud Agents are set on the test. Source ports are numbers above 1024 chosen by the operating system's networking stack.
Last updated