Data Collected by Endpoint Agent

This article covers the data collected by ThousandEyes Endpoint Agent.

Website Visits

While Inside a Monitored Network

If the ThousandEyes Endpoint Agent browser extension (for Chrome, Edge or IE) is installed and enabled, it monitors all page navigations within a non-incognito session. This extension extracts the following metadata (which is then displayed on the ThousandEyes application):

• Per Page

  • Title

  • Request URL (e.g., https://example.com)

  • Request Timings (e.g., DNS resolution time, connect time)

  • Page Load Timings (e.g. page load time)

  • Response Status Code

• Per Page Resource

  • Request URL (e.g., https://example.com/static/image.jpg)

  • Request Timings

  • Response Status Code

  • Request/Response Cookies

  • Request/Response Headers

  • Request/Response Headers Size

  • Request/Response Body Size

Also the above metadata payload does not capture the content (body) of any request/response. Therefore, the extension will never capture nor forward the following:

• JSON/HTML/etc data sent with POST requests

• JSON/HTML/etc data received by responses

• X-www-form-urlencoded input data sent by form submissions

The following sensitive request/response header values are explicitly omitted (not captured) by the extension:

• Set-Cookie

• Cookie

• Authorization

While Outside a Monitored Network

No data is automatically collected while an Endpoint Agent communicates with ThousandEyes from outside a monitored network. Manual recording may still be initiated by the endpoint user, targeting ANY (monitored or unmonitored) domain. These results will be collected and reported back to ThousandEyes and will appear in the Endpoint Agent views.

Web Performance Data (Web and Session Details Views)

Web Performance data includes HTTP Archive (HAR) format data. HAR information collected by the Endpoint Agent includes each file accessed on a particular site, and includes request and response header information, timing, source and destination IP addresses, as well as wait and receive timing for each component loaded in each page visited. Sensitive information in headers (such as cookie information and authorization data) is suppressed at collection time.

For more information on the content of HAR format data, refer to http://www.softwareishard.com/blog/firebug/http-archive-specification/

Waterfall data is shown for each page visited. A “session” constitutes either:

• a user visit to a domain, using a specific protocol (ie, http://www.google.com and https://www.google.com would be separate sessions, since the protocol differs between the two domains, however, multiple subsequent visits to https://www.google.com would be recorded in the same session)
• a manual recording initiated by an endpoint user - the session will last from the first page that the user clicks the record button through the last page of the recording.

A waterfall (“page”) will be captured each time the DOM is reloaded in a session (ie, navigation to another page, form submission and/or page refresh). Multiple pages can be shown in a single session.

Network Data (Network and Session Details Views)

Network data is collected in a number of different ways, and differs from data captured by the ThousandEyes Enterprise Agent. A detailed list of each series of packets sent is shown below.

Network probes are done in the following ways:

• ICMP ping: Sends 10 ICMP packets with 1 second interval. The round trip time (RTT) is captured and the sent/received ratio.

• ICMP path trace: Performs an ICMP-based TTL path trace with a maximum of 32 hops. Information about each hop is captured, including RTT. If the Endpoint Agent is running on a Mac OS X client, MPLS information will also be captured and shown

• TCP connect: Opens a TCP connection with a 10 second timeout and closes the connection if it was able to connect. Timing, and error code (if applicable) is captured.

• TCP ping: This probe conceptually does not differ much from the ICMP-based one - the difference in the implementation can be found here.
• TCP path trace: The network measurement done in this probe is similar to the ICMP-based TTL path trace but the differences can be found here.

Network Data for Scheduled Tests

The configuration option (protocol) determines what network probes are sent to collect the network data:

Protocol	ICMP ping	ICMP path trace	TCP connect	TCP ping	TCP path trace
ICMP+TCP connect	Yes	Yes	Yes
Prefer TCP (agent supports TCP)				Yes	Yes
Prefer TCP (agent does not support TCP)	Yes	Yes	Yes
Driver(required)				Yes	Yes

Network Data for Local Networks

Based on the connection topology of the Endpoint, network probes will be sent to the following destinations:

Target	ICMP ping	ICMP path trace	TCP connect
Gateway	X
Destination	X	X	X
Proxy (if used)	X	X	X
VPN (if used)	X	X

Computer Information (Session Details View)

Some information about the computer where the Endpoint Agent is installed is collected as well.

Field	Description	Example
Platform	Base operating system	Windows / Mac
OS Version	Major/minor version of operating system	Microsoft Windows 8.1 Enterprise
Kernel Version	Kernel version numbers	6.3.9600
Browser	Browser used for data collection	Google Chrome (46.0.2490.80)
Endpoint Version	Major/minor version of endpoint agent	0.24.1
IP Address	Private IP address	10.1.1.100
DNS Servers	Addresses of configured DNS server	10.1.1.253, 10.1.1.254
Manufacturer	Hardware manufacturer	Lenovo
Model	Hardware model	20ARS18N00
Memory	Total memory available to Operating System	8192 MB
Computer Name	Computer Name	win81-1
Logged in user	User name	boulders\dave

Network Information (Session Details and Network Topology Views)

In addtion to the computer information, the following network information is collected

Field	Description	Example
Network Name	Name of wireless network	BOULDERS
BSSID	Base Station ID (mac address)	8e:2f:44:4a:ae:bf
Channel	Wireless channel	2 (2.4 GHz)
Signal Strength	Signal strength (dBm)	-32
Signal Quality	Signal quality (expressed as a percentage)	99%
Transmission Rate	Maximum transmission as seen by operating system	130 Mbps
Physical Mode	IEEE 802.11 specification for wireless connection	802.11n
Hardware type	Connection type (Wired/Wireless)	Wireless
Proxy method	If a proxy is used, the method (PAC file, WPAD, manual)	Network PAC Script
Proxy configuration URL	If proxy autoconfiguration is used, the URL where the file is sourced from	http://10.1.1.1/scripts/autoproxy.pac
Network Gateway	Default network gateway	10.1.1.1

Local Network Information (Network Access and Wireless)

The following data is continuously collected while the endpoint is active:

Metric	Description
DNS Server Loss	The percentage measurement of lost ICMP Echo Reply packets from the DNS server out of the total ICMP Echo Request packets sent.
DNS Server Latency	The Latency to the DNS server
Domain Resolution Time	The Domain Resolution Time is equivalent to the time it takes the endpoint agent to query the primary DNS server for the root domain.
Gateway Loss	The percentage of packet loss to connect to the gateway
Gateway Latency	The latency to the default gateway
Link Speed	The link speed identified by the OS
Wireless Signal Quality	The signal quality identified by the OS
Proxy Loss	The percentage of packet loss to the Explicit Proxy connection
Proxy Latency	Latency to the Explicit Proxy connection
CPU Load	The CPU load provided the OS (percentage)
Memory	Memory usage provided by the OS (percentage)
VPN Loss	The percentage of packet loss to the VPN gateway
VPN Latency	Latency to the VPN gateway
Signal Quality	Wi-Fi signal quality reported by the OS
Throughput	Calculated based on bytes transferred during the test round
Retransmission Rate*	The percentage of retransmission rate reported by the OS
Roaming Events	The number of roaming events reported by the OS
Channel Swap Events	The number of channel swap events reported by the OS

**Retransmisssion Rate** - The retransmission rate counter is not supported for Endpoint Agents installed on macOS as the information needed to calculate a retransmission rate is not publicly available on macOS.