ThousandEyes provides a full suite of terminal based Network, DNS, and Voice utilities. These utilities come from the same code base that our Agents use to perform tests, and provide enhanced insight during troubleshooting.
Installing CLI Network Troubleshooting Utilities
The ThousandEyes Utilites Suite is bundled within the te-agent-utils package and is available for any host running a supported Linux distribution.
ThousandEyes repositories are automatically added to a host's package manager during Enterprise Agent installation. Hosts without an Enterprise Agent will require manual installation of a ThousandEyes software package repository.
A replacement for traditional ping, and can be run in a number of different application protocol modes. te-ping includes options to run in SACK, SYN, ICMP and VoIP modes, designed for use with different types of targets, and is consistent with the way that ThousandEyes measures end-to-end metrics in Network tests. Outputs include Loss, Latency and Jitter. Requires sudo permissions.
Syntax and Options
(sudo) te-ping [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test, do not use in icmp mode
<random value>
-p
--port
destination port for the probe, do not use in icmp mode
80
-c
--count
number of packets to be sent by the test
50
-I
--interval
initial maximum interval between packets (in ms)
40
-i
--interface
interface name to use for the test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address (use in conjunction with the --interface option)
n/a
-M
--mode
transmission mode for the test; accepts sack, syn, icmp, rtp (UDP+RTP), or sip options
sack
-v
--verbose
verbosity of the test; short command accepts -v (verbose) or -vv (very verbose), -vvv (very very verbose)
n/a
Example
$ sudo te-ping -i eth0 74.125.225.176
--- 74.125.225.176 ping statistics ---
50 packets transmitted, 50 received, 0.0% packet loss
rtt min/avg/max/mdev = 68.00/69.42/71.00/0.58 ms
$ sudo te-ping -c 25 thousandeyes.com
--- 54.215.17.122 ping statistics ---
25 packets transmitted, 25 received, 0.0% packet loss
rtt min/avg/max/mdev = 5.00/5.68/6.00/0.44 ms
te-pathtrace
Runs a traceroute, using the same methods as are used by ThousandEyes to the endpoint. In addition to obtaining the IP addresses transited by each probe, our code shows quoted-ttl, response-ttl and MPLS label values for each probe. Since each agent test runs 3 probes (test attempts) to the target, if you want consistent results with that of the agent, run with option -P 3, to obtain three sets of trace data. Requires sudo permissions. Note in the example shown that the path to the same target IP diverges in two separate trace requests from the same target, at hop # 4 (shown in bold for emphasis). Using the Paris-traceroute algorithm (which is used by ThousandEyes in our Path Visualization view), this would show a path split, as in the image shown below.
Syntax and Options
(sudo) te-pathtrace [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test, do not use in icmp mode
<random value>
-p
--port
destination port for the probe, do not use in icmp mode
80
-w
--wait-time
how long to wait for each probe reply (in ms)
1000
-g
--gap-limit
maximum number of consecutive unresponsive hops before stopping
6
-l
--max-loops
number of times the same IP can be encountered before stopping
2
-P
--paths
number of parallel paths to trace
1
-i
--interface
network interface to use for this test (use in conjunction with --source-ip option)
any
-M
--mode
transmission mode to use for the test; accepts tcp, sack, icmp, rtp, or sip.
Designed to quickly interrogate the path between your test endpoint and the target. The command will find the smallest allowed MTU on the path between the endpoints, allowing inference of certain types of tunnels, or misconfigurations of MTU size on routers between the endpoints. By default, the MTU is 1500 bytes.
Syntax and Options
(sudo) te-pathmtu [options] <target-ip>
Parameter
Description
target-ip
IP address of the target machine or DNS name of target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test, do not use in icmp mode
<random value>
-p
--port
destination port for the probe, do not use in icmp mode
80
-w
--wait-time
how long to wait for each probe reply (in ms)
1000
-m
--max-mtu
the maximum possible transmission unit of the output interface (in bytes)
1500
-P
--paths
number of parallel paths to trace
1
-i
--interface
network interface to use for this test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address ( use in conjunction with --interface option)
n/a
-M
--mode
transmission mode to use for the test; accepts tcp, icmp, rtp, or sip
Looking up path MTU from 89.187.112.188 to 98.139.183.24:80
Path MTU: 1500
te-bw
Designed as a replacement for iPerf, te-bw allows measurement of bandwidth, while having control of a single endpoint, and without the need to have exclusive access to the pipe. Use te-bw to measure estimated capacity, and available bandwidth on the pipe between the endpoint and target. Requires sudo permissions. When measuring bandwidth, te-bw shows the limiting bandwidth for UPSTREAM communication from the endpoint (ability of the endpoint to send data to the target), and DOWNSTREAM bandwidth from the target (ability for the target to send data to the endpoint). This is an important point for servers behind asymmetric links, such as ADSL.
Syntax and Options
(sudo) te-bw [options] <target-ip>
Parameter
Description
target-ip
IP address or DNS name of the target machine.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
-p
--port
destination port on the target. Do not use in icmp mode
<random value>
-j
--jitter
mean deviation of the round trip time (in μs)
if not specified, will be calculated by running an initial ping against the target
-z
--probe-size
maximum size of the probe packets (in bytes)
1440
-c
--capacity
capacity of the link (in Mbps), 0=unlimited.
if not specified, will be calculated by running an initial calculation against the target
-s
--source-ip
source IP address ( use in conjunction with --interface option)
n/a
-T
--timeout
timeout for the test to complete (in secs) , 0 = unlimited
10
-v
--verbose
verbosity level, short option accepts -v (verbose), -vv (very verbose), -vvv (very very verbose)
n/a
-i
--interface
network interface to use for the test (use in conjunction with --source-ip option)
any
-M
--mode
transmission mode to use for the test; accepts icmp or tcp
tcp
Example
$ sudo te-bw -i eth0 192.168.1.209 74.125.225.176
Running ping task to estimate jitter
80 bytes from 74.125.225.176: tcp_req=1 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=2 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=3 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=4 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=5 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=6 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=7 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=8 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=9 ttl=52 time=115 ms
80 bytes from 74.125.225.176: tcp_req=10 ttl=52 time=115 ms
--- 74.125.225.176 ping statistics ---
10 packets transmitted, 10 received, 0.0% packet loss
rtt min/avg/max/mdev = 115.00/115.00/115.00/0.00 ms
Capacity
========================================
Estimated rate: 96.1 Mbps +/-0
Data sent: 8.80 KB
Probe size: 1420 B
Wall time: 3.59 secs
PCap stats: 200/0 pkts
Return code: 0 ()
========================================
Available bandwidth
========================================
Estimated rate: 78.0 Mbps +/-8
Data sent: 50.60 KB
Probe size: 1420 B
Wall time: 4.34 secs
PCap stats: 1150/0 pkts
Return code: 0 ()
========================================
DNS Utilities
te-dns-trace
A special version of dig +trace which follows CNAME records across zones, in order to get to the requested record type. This is the same logic as is followed using the DNS Domain Trace test inside ThousandEyes. In addition, you'll receive more verbose feedback when the trace attempt is unsuccessful at retrieving records from an authoritative server, before the request fails over to a subsequent server. In the example shown below for www.healthcare.gov A, notice how the trace is followed from CNAME retrieved from the .gov zone, to Akamai, where the ultimate A record resides. A standard dig +trace request would stop at the line shown in bold. Does not require sudo permissions.
Syntax and Options
(sudo) te-dns-trace <domain> [type]
Parameter
Description
Required?
Default Value
domain
domain name to look up
Y
n/a
type
DNS record type
N
A
Example
$ sudo te-dns-trace www.healthcare.gov A
gov. 172800 IN NS b.gov-servers.net.
gov. 172800 IN NS a.gov-servers.net.
;; Received 143 bytes from 198.41.0.4(a.root-servers.net.) in 25 ms
healthcare.gov. 86400 IN NS eur5.akam.net.
healthcare.gov. 86400 IN NS usc3.akam.net.
healthcare.gov. 86400 IN NS use2.akam.net.
healthcare.gov. 86400 IN NS use4.akam.net.
healthcare.gov. 86400 IN NS usw2.akam.net.
healthcare.gov. 86400 IN NS asia1.akam.net.
healthcare.gov. 86400 IN NS ns1-78.akam.net.
healthcare.gov. 86400 IN NS ns1-252.akam.net.
;; Received 202 bytes from 209.112.123.30(b.gov-servers.net.) in 23 ms
www.healthcare.gov. 120 IN CNAME www.geodirector.hc.gov.akadns.net.
;; Received 83 bytes from 23.74.25.64(eur5.akam.net.) in 72 ms
net. 172800 IN NS m.gtld-servers.net.
net. 172800 IN NS l.gtld-servers.net.
net. 172800 IN NS k.gtld-servers.net.
net. 172800 IN NS j.gtld-servers.net.
net. 172800 IN NS i.gtld-servers.net.
net. 172800 IN NS h.gtld-servers.net.
net. 172800 IN NS g.gtld-servers.net.
net. 172800 IN NS f.gtld-servers.net.
net. 172800 IN NS e.gtld-servers.net.
net. 172800 IN NS d.gtld-servers.net.
net. 172800 IN NS c.gtld-servers.net.
net. 172800 IN NS b.gtld-servers.net.
net. 172800 IN NS a.gtld-servers.net.
;; Received 508 bytes from 198.41.0.4(a.root-servers.net.) in 27 ms
akadns.net. 172800 IN NS ns1-129.akadns.net.
akadns.net. 172800 IN NS ns2-129.akadns.net.
akadns.net. 172800 IN NS ns17-133.akadns.org.
akadns.net. 172800 IN NS a3-129.akadns.net.
akadns.net. 172800 IN NS a11-129.akadns.net.
akadns.net. 172800 IN NS a9-128.akadns.net.
akadns.net. 172800 IN NS a5-130.akadns.org.
akadns.net. 172800 IN NS a4-131.akadns.org.
akadns.net. 172800 IN NS a10-128.akadns.org.
akadns.net. 172800 IN NS a28-129.akadns.org.
;; Received 358 bytes from 192.55.83.30(m.gtld-servers.net.) in 80 ms
www.geodirector.hc.gov.akadns.net. 120 IN CNAME production.healthcare.gov.edgekey.net.
;; Received 99 bytes from 193.108.88.129(ns1-129.akadns.net.) in 72 ms
edgekey.net. 172800 IN NS ns1-66.akam.net.
edgekey.net. 172800 IN NS usw6.akam.net.
edgekey.net. 172800 IN NS use9.akam.net.
edgekey.net. 172800 IN NS adns1.akam.net.
edgekey.net. 172800 IN NS adns2.akam.net.
edgekey.net. 172800 IN NS adns3.akam.net.
edgekey.net. 172800 IN NS adns4.akam.net.
edgekey.net. 172800 IN NS ns4-66.akam.net.
edgekey.net. 172800 IN NS ns4-67.akam.net.
edgekey.net. 172800 IN NS ns7-65.akam.net.
edgekey.net. 172800 IN NS ns7-66.akam.net.
edgekey.net. 172800 IN NS ns5-66.akam.net.
edgekey.net. 172800 IN NS ns5-64.akam.net.
;; Received 497 bytes from 192.55.83.30(m.gtld-servers.net.) in 81 ms
production.healthcare.gov.edgekey.net. 21600 IN CNAME e8132.dscb.akamaiedge.net.
;; Received 91 bytes from 193.108.91.66(ns1-66.akam.net.) in 23 ms
akamaiedge.net. 172800 IN NS la1.akamaiedge.net.
akamaiedge.net. 172800 IN NS la6.akamaiedge.net.
akamaiedge.net. 172800 IN NS la7.akamaiedge.net.
akamaiedge.net. 172800 IN NS la3.akamaiedge.net.
akamaiedge.net. 172800 IN NS lar6.akamaiedge.net.
akamaiedge.net. 172800 IN NS lar2.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns3-194.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns2-194.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns4-194.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns6-194.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns7-194.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns5-194.akamaiedge.net.
akamaiedge.net. 172800 IN NS ns1-1.akamaiedge.net.
;; Received 497 bytes from 192.55.83.30(m.gtld-servers.net.) in 80 ms
dscb.akamaiedge.net. 21600 IN NS n3dscb.akamaiedge.net.
dscb.akamaiedge.net. 43200 IN NS n2dscb.akamaiedge.net.
dscb.akamaiedge.net. 32400 IN NS n4dscb.akamaiedge.net.
dscb.akamaiedge.net. 32400 IN NS n7dscb.akamaiedge.net.
dscb.akamaiedge.net. 32400 IN NS n1dscb.akamaiedge.net.
dscb.akamaiedge.net. 21600 IN NS n6dscb.akamaiedge.net.
dscb.akamaiedge.net. 43200 IN NS n5dscb.akamaiedge.net.
dscb.akamaiedge.net. 21600 IN NS n0dscb.akamaiedge.net.
;; Received 339 bytes from 184.26.161.192(la1.akamaiedge.net.) in 23 ms
e8132.dscb.akamaiedge.net. 20 IN A 23.43.186.194
;; Received 59 bytes from 63.217.8.179(n3dscb.akamaiedge.net.) in 40 ms
te-sigchase
Provides DNSSEC keychain validation (using bottom-up method) for the target record type. Requires that DNSSEC extensions are supported by routers used by the endpoint. See http://www.icann.org/en/groups/ssac/documents/sac-035-en.pdf for a list of devices which do not currently support required extensions for DNSSEC compatibility. Outputs of DNSSEC keychain validation will be the Data Chain, and Trust Tree responses for the validation request, similar to that which is shown in the output of DNSSEC tests in ThousandEyes. Does not require sudo permissions.
Syntax and Options
(sudo) te-sigchase <domain> [type] [key-file]
Parameter
Description
Required?
Default Value
domain
domain name to validate
Y
n/a
type
record type to validate
N
A
key-file
key file used for validation of root server signatures
N
A key file is provided and maintained with the te-agent-utils package. To view the content of the file, look at /var/lib/te-agent-utils/dns-rootserver.key
Designed to initiate an instant voice call to a VoIP server. It includes options to choose different Differentiated Services Code Point (DSCP) and codec values. Voice packets are encapsulated by a Real Time Transport Protocol (RTP) header, followed by further encapsulation through the transport layer protocol UDP and then sent using IP
Syntax and Options
(sudo) te-rtp [options] <target-ip>
Parameter
Description
target-ip
IP address of the VoIP server.
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--source-port
source port to be used for the test
<random value>
-p
--port
destination port for the probe
5000
-P
--paths
# of paths to trace (in parallel)
1
-i
--interface
interface name to use for the test (use in conjunction with --source-ip option)
any
-s
--source-ip
source IP address ( use in conjunction with --interface option)
n/a
--dscp
DSCP value to be used for the voice call
0 = best effort
--codec
Audio codec value used for the VoIP call, 0=G.711, 1=G.722_24, 2=G.722_32
0 = G.711
-d
--stream-duration
To run the voice test call for desired time in seconds. For each 5 seconds, 50 packets are sent to the server
5 seconds
-o
--clock-offset
clock offset with the server
0
-v
--verbose
verbosity of the test; short command accepts -v (verbose) or -vv (very verbose), -vvv (very very verbose)
n/a
te-server
Designed to open up a port on the local machine to allow communication with the VoIP client. The desired server port should be configured to accept UDP connections inbound. It is always the machine running the te-voipserver utility that logs the probe statistics.
Syntax and Options
(sudo) te-server [options] <local-port>
Parameter
Description
local-port
Port on which the sever listens for incoming VoIP connections (UDP traffic)
Option (short)
Option (long)
Description
Default Value
-h
--help
displays usage instructions
n/a
--stunner-host
host in which to reach the STUNNER server
--stunner-port
port in which to reach the STUNNER server
-6
--ipv6
use IPv6 instead of IPv4
n/a
-j
--jitter
the size of the de-jitter buffer
0.1000000001 secs
--check-nat
whether to check if the NAT allows hole punching (only valid when using --stunner-host)
1
--agent-id
the agent id to use when connecting and authenticating to the STUNNER server
0
-v
--verbose
verbosity of the test; short command accepts -v (verbose) or -vv (very verbose), -vvv (very very verbose)
n/a
Example
user@ubuntu:~$ sudo te-server 49152
13 [0x7fab82d4c700] INFO te.pte null - ProbeTaskExecutor started.
65 [0x7fab82d4c700] INFO te.port_49152#1 null - port_49152 server task started.
[user1@centos]$ sudo te-rtp 10.100.10.166 -p 49152
241 packets sent in 5000ms @96% of rate
user@ubuntu:~$ sudo te-voipserver 49152
13 [0x7fab82d4c700] INFO te.pte null - ProbeTaskExecutor started.
Server listening at port 49152 on interface any
65 [0x7fab82d4c700] INFO te.port_49152#1 null - Voip server receiving streams at 0.0.0.0:49152
44435 [0x7fab82d4c700] INFO te.port_49152#1 null - Task/Client 0/0: round #1407879622 (G.711_64kbps): 482/0/0 (total/lost/discarded frames); wall time: 5001 @ 98% of desired rate; MOS: 4.4; OWD: 0.0; jitter(RTCP/PDV): 0.0/0.0;
Note: The wall time accounts for the total time taken for the test to complete including the programmatically added time and/or time taken for the resources to be made available.
