CLI Network Troubleshooting Utilities
ThousandEyes provides a full suite of terminal based Network, DNS, and Voice utilities. These utilities come from the same code base that our Agents use to perform tests, and provide enhanced insight during troubleshooting.
Installing CLI Network Troubleshooting Utilities
The ThousandEyes Utilites Suite is provided by the te-agent-utils package and is available for any host running a supported Linux distribution.
ThousandEyes repositories are automatically added to a host's package manager during Enterprise Agent installation. Hosts without an Enterprise Agent will require manual installation of a ThousandEyes software package repository.
Installation for Ubuntu
Install the ThousandEyes APT Repository
Add the ThousandEyes repository
Add the ThousandEyes signing key
Changes will be applied during the update process
If the key has been properly added it will be listed as a trusted key
Install the te-agent-utils Software Package
Installation for YUM (RedHat, Rocky, Amazon, CentOS, and Oracle Linux)
Install the ThousandEyes YUM Repository
Add the applicable ThousandEyes repository
Add the ThousandEyes signing key
Changes will be applied during the update process
Check the GPG key
The ThousandEyes key fingerprint should be as follows
Install the te-agent-utils Software Package
Network Utilities
te-ping
A replacement for traditional ping, and can be run in a number of different application protocol modes. te-ping includes options to run in SACK, SYN, ICMP and VoIP modes, designed for use with different types of targets, and is consistent with the way that ThousandEyes measures end-to-end metrics in Network tests. Outputs include Loss, Latency and Jitter. Requires sudo permissions.
Syntax and Options
(sudo) te-ping [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test, do not use in icmp mode
<random value>
-p
--port
destination port for the probe, do not use in icmp mode
80
-c
--count
number of packets to be sent by the test
50
-I
--interval
initial maximum interval between packets (in ms)
40
-i
--interface
interface name to use for the test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address (use in conjunction with the --interface option)
n/a
-M
--mode
transmission mode for the test; accepts sack, syn, icmp, rtp (UDP+RTP), or sip options
sack
-v
--verbose
verbosity of the test; short command accepts -v (verbose) or -vv (very verbose), -vvv (very very verbose)
n/a
Example
te-pathtrace
Runs a traceroute, using the same methods as are used by ThousandEyes to the endpoint. In addition to obtaining the IP addresses transited by each probe, our code shows quoted-ttl, response-ttl and MPLS label values for each probe. Since each agent test runs 3 probes (test attempts) to the target, if you want consistent results with that of the agent, run with option -P 3, to obtain three sets of trace data. Requires sudo permissions. Note in the example shown that the path to the same target IP diverges in two separate trace requests from the same target, at hop # 4 (shown in bold for emphasis). Using the Paris-traceroute algorithm (which is used by ThousandEyes in our Path Visualization view), this would show a path split, as in the image shown below.
Syntax and Options
(sudo) te-pathtrace [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test, do not use in icmp mode
<random value>
-p
--port
destination port for the probe, do not use in icmp mode
80
-w
--wait-time
how long to wait for each probe reply (in ms)
1000
-g
--gap-limit
maximum number of consecutive unresponsive hops before stopping
6
-l
--max-loops
number of times the same IP can be encountered before stopping
2
-P
--paths
number of parallel paths to trace
1
-M
--mode
transmission mode to use for the test; accepts tcp, sack, icmp, rtp, or sip.
tcp
-v
--verbose
verbosity level, accepts -v (verbose), -vv (very verbose), -vvv (very very verbose)
n/a
Example
te-pathmtu
Designed to quickly interrogate the path between your test endpoint and the target. The command will find the smallest allowed MTU on the path between the endpoints, allowing inference of certain types of tunnels, or misconfigurations of MTU size on routers between the endpoints. By default, the MTU is 1500 bytes.
Syntax and Options
(sudo) te-pathmtu [options] <target-ip>
Parameter
Description
target-ip
IP address of the target machine or DNS name of target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test, do not use in icmp mode
<random value>
-p
--port
destination port for the probe, do not use in icmp mode
80
-w
--wait-time
how long to wait for each probe reply (in ms)
1000
-m
--max-mtu
the maximum possible transmission unit of the output interface (in bytes)
1500
-P
--paths
number of parallel paths to trace
1
-i
--interface
network interface to use for this test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address ( use in conjunction with --interface option)
n/a
-M
--mode
transmission mode to use for the test; accepts tcp, icmp, rtp, or sip
tcp
-v
--verbose
verbosity level, accepts -v (verbose), -vv (very verbose) -vvv (very very verbose)
n/a
Example
te-bw
Designed as a replacement for iPerf, te-bw allows measurement of bandwidth, while having control of a single endpoint, and without the need to have exclusive access to the pipe. Use te-bw to measure estimated capacity, and available bandwidth on the pipe between the endpoint and target. Requires sudo permissions. When measuring bandwidth, te-bw shows the limiting bandwidth for UPSTREAM communication from the endpoint (ability of the endpoint to send data to the target), and DOWNSTREAM bandwidth from the target (ability for the target to send data to the endpoint). This is an important point for servers behind asymmetric links, such as ADSL.
Syntax and Options
(sudo) te-bw [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
-p
--port
destination port on the target. Do not use in icmp mode
<random value>
-j
--jitter
mean deviation of the round trip time (in μs)
if not specified, will be calculated by running an initial ping against the target
-z
--probe-size
maximum size of the probe packets (in bytes)
1440
-c
--capacity
capacity of the link (in Mbps), 0=unlimited.
if not specified, will be calculated by running an initial calculation against the target
-s
--source-ip
source IP address ( use in conjunction with --interface option)
n/a
-T
--timeout
timeout for the test to complete (in secs) , 0 = unlimited
10
-v
--verbose
verbosity level, short option accepts -v (verbose), -vv (very verbose), -vvv (very very verbose)
n/a
-i
--interface
network interface to use for the test (use in conjunction with --source-ip option)
any
-M
--mode
transmission mode to use for the test; accepts icmp or tcp
tcp
Example
te-oneway
te-oneway performs an Agent-to-Agent test in a single direction.
Syntax and Options
(sudo) te-oneway [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target agent.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
-v
--verbose
verbosity level, short option accepts -v (verbose), -vv (very verbose), -vvv (very very verbose)
--stunner-host
host in which to reach the STUNNER server
--stunner-port
port in which to reach the STUNNER server
-p
--port
destination probe port on the target agent
5000
-i
--interface
network interface to use for the test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address (use in conjunction with --interface option)
any
--dscp arg
DSCP value (decimal), 0=best effort
0
--payload-size
payload size to use
56
-M
--mode
mode: tcp or udp
tcp
-b
--behind-nat
behave as if this client is behind a NAT (makes sense when using STUNNER)
--agent-id
the agent id to use when connecting and authenticating to the STUNNER
--agent-token
the agent token to use when connecting and authenticating to the STUNNER
--packet-rate
Rate at which to send packets (in pps), or 0 for RTT-based mechanism
0
-a
--asymmetric-interfaces
Support separate ingress/egress interfaces
te-stresstest
te-stresstest runs a suite of tests in parallel against a single endpoint. The goal is to simulate multiple agents running tests against one endpoint.
Syntax and Options
(sudo) te-stresstest [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
produce this help message
-p
--port
the destination probe port (in TCP mode)
80
--ping
number of concurrent ping tasks
0
--pathtrace
number of concurrent pathtrace tasks
0
--pathmtu
number of concurrent pathmtu tasks
0
--bw
number of concurrent available bandwidth tasks (runs capacity estimation too if --capacity is not specified)
0
--voice
number of concurrent voice tasks
0
-M
--mode
mode: tcp, icmp, or rtp
tcp
-g
--gap-limit
number of unresponsive hops before stopping ('auto' to perform distance estimation to destination, in which case it autouses 20 estimated_distance or 7 if destination doesn't reply)
auto
-j
--jitter
mean dev of two-way jitter (in usecs)
-z
--probe-size
maximum size of probe packets (in bytes)
1440
-c
--capacity
capacity of the link (in Mbps, 0=unlimited)
0
--dscp
DSCP value (decimal), 0=best effort
0
--codec
audio codec, 0=G711, 1=G722_24, 2=G722_32, 3=G726, 4=G723, 5=G729a
0
-o
--clock-offset
clock offset with server (in secs)
0
-t
--threads
# of threads
1
--voice-threads
runs voice tasks in a dedicated probe task executor (boost asio queue) with n threads (0 means that voice tasks run in shared probe task executor; (more than 0 means that voice tasks run in a dedicated probe task executor with that many threads)
0
-d
--duration
duration of the test (in secs, 0=unlimited)
10
-T
--timeout
deadline for individual tasks to complete (in ms)
5
-i
--interface
interface (use in conjunction with --source-ip option)
-s
--source-ip
source IP address (use in conjunction with --interface option)
-v
--verbose
verbosity level: -v, -vv, or -vvv
te-tput-tcp
te-tput-tcp runs a throughput test to an Enterprise Agent. The test will send as much data as it can in the set amount of time to measure the maximum data throughput to an agent.
Syntax and Options
(sudo) te-tput-tcp [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
produce this help message
-v
--verbose
verbosity level: -v, -vv, or -vvv
--source-port
the source probe port, 0=random
0
-p
--port
the destination probe port
5000
-i
--interface
interface (use in conjunction with --source-ip option)
-s
--source-ip
source IP address (use in conjunction with --interface option)
--dscp
DSCP value (decimal), 0=best effort
0
--agent-id
the agent id to use when connecting and authenticating to the STUNNER
0
--mss
mss to use
1460
--send-buffer
send buffer size to use
0
--no-delay
use no delay on socket (Nagle)
0
--congestion-algo
the TCP congestion algorthm to use for setsockopt TCP_CONGESTION
-d
--stream-duration
stream duration (in secs)
5
-a
--asymmetric-interfaces
Support separate ingress/egress interfaces
te-tput-udp
te-tput-udp runs a throughput test to an Enterprise Agent, similar to te-tput-tcp but using UDP rather than TCP.
Syntax and Options
(sudo) te-tput-udp [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
produce this help message
-v
--verbose
verbosity level: -v, -vv, or -vvv
--source-port
the source probe port, 0=random
0
-p
--port
the destination probe port
5000
-i
--interface
interface (use in conjunction with --source-ip option)
-s
--source-ip
source IP address (use in conjunction with --interface option)
--dscp
DSCP value (decimal), 0=best effort
0
--agent-id
the agent id to use when connecting and authenticating to the STUNNER
0
--payload-size
paylod size to use
1400
-d
--stream-duration
stream duration (in secs)
5
-r
--rate
rate in kbps
10240
-a
--asymmetric-interfaces
Support separate ingress/egress interfaces
DNS Utilities
te-dns-trace
A special version of dig +trace which follows CNAME records across zones, in order to get to the requested record type. This is the same logic as is followed using the DNS Domain Trace test inside ThousandEyes. In addition, you'll receive more verbose feedback when the trace attempt is unsuccessful at retrieving records from an authoritative server, before the request fails over to a subsequent server. In the example shown below for www.healthcare.gov A, notice how the trace is followed from CNAME retrieved from the .gov zone, to Akamai, where the ultimate A record resides. A standard dig +trace request would stop at the line shown in bold. Does not require sudo permissions.
Syntax and Options
(sudo) te-dns-trace <domain> [type]
Parameter
Description
Required?
Default Value
domain
domain name to look up
Y
n/a
type
DNS record type
N
A
Example
te-sigchase
Provides DNSSEC keychain validation (using bottom-up method) for the target record type. Requires that DNSSEC extensions are supported by routers used by the endpoint. See http://www.icann.org/en/groups/ssac/documents/sac-035-en.pdf for a list of devices which do not currently support required extensions for DNSSEC compatibility. Outputs of DNSSEC keychain validation will be the Data Chain, and Trust Tree responses for the validation request, similar to that which is shown in the output of DNSSEC tests in ThousandEyes. Does not require sudo permissions.
Syntax and Options
(sudo) te-sigchase <domain> [type] [key-file]
Parameter
Description
Required?
Default Value
domain
domain name to validate
Y
n/a
type
record type to validate
N
A
key-file
key file used for validation of root server signatures
N
A key file is provided and maintained with the te-agent-utils package. To view the content of the file, look at /var/lib/te-agent-utils/dns-rootserver.key
Example
Voice Utilities
te-rtp
Designed to initiate an instant voice call to a VoIP server. It includes options to choose different Differentiated Services Code Point (DSCP) and codec values. Voice packets are encapsulated by a Real Time Transport Protocol (RTP) header, followed by further encapsulation through the transport layer protocol UDP and then sent using IP
Syntax and Options
(sudo) te-rtp [options] <target-ip>
Parameter
Description
target-ip
IP address of the VoIP server.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test
<random value>
-p
--port
destination port for the probe
5000
-P
--paths
# of paths to trace (in parallel)
1
-i
--interface
interface name to use for the test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address ( use in conjunction with --interface option)
n/a
--dscp
DSCP value to be used for the voice call
0 = best effort
--codec
Audio codec value used for the VoIP call, 0=G.711, 1=G.722_24, 2=G.722_32
0 = G.711
-d
--stream-duration
To run the voice test call for desired time in seconds. For each 5 seconds, 50 packets are sent to the server
5 seconds
-o
--clock-offset
clock offset with the server
0
-v
--verbose
verbosity of the test; short command accepts -v (verbose) or -vv (very verbose), -vvv (very very verbose)
n/a
te-server
Designed to open up a port on the local machine to allow communication with the VoIP client. The desired server port should be configured to accept UDP connections inbound. It is always the machine running the te-voipserver utility that logs the probe statistics.
Syntax and Options
(sudo) te-server [options] <local-port>
Parameter
Description
local-port
Port on which the sever listens for incoming VoIP connections (UDP traffic)
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--stunner-host
host in which to reach the STUNNER server
--stunner-port
port in which to reach the STUNNER server
-6
--ipv6
use IPv6 instead of IPv4
n/a
-j
--jitter
the size of the de-jitter buffer
0.1000000001 secs
--check-nat
whether to check if the NAT allows hole punching (only valid when using --stunner-host)
1
--agent-id
the agent id to use when connecting and authenticating to the STUNNER server
0
-v
--verbose
verbosity of the test; short command accepts -v (verbose) or -vv (very verbose), -vvv (very very verbose)
n/a
Example
Note: The wall time accounts for the total time taken for the test to complete including the programmatically added time and/or time taken for the resources to be made available.
Last updated
