Firewall Configuration for Enterprise Agents

When installing a ThousandEyes Enterprise Agent behind a firewall or similar device (such as a router with access control lists (ACLs)), the device must be configured with rules that allow the Enterprise Agent to register with the ThousandEyes platform, execute tests, report test results, and access necessary infrastructure services such as the Domain Name Service (DNS), the Network Time Protocol (NTP) and repositories for software package updates.

This article provides a complete set of information to allow Enterprise Agent network communication to traverse a firewall or similar device. For administrators wishing to quickly install an Enterprise Agent, review the Base Rules section for the instructions required to register the agent in the ThousandEyes platform. Software updates are covered in the Installation Type Rules section.

Introduction

A firewall rule or ACL for Enterprise Agent communication is specified using one or more of the following criteria:

  • Destination IP address(es) or DNS domain name(s)

  • Destination Port numbers (if the protocol is TCP or UDP)

  • Protocol (TCP, UDP, or ICMP)

  • Direction (outbound from the agent unless otherwise noted)

Notes:

  1. To use domain names in rules or ACLs, the firewall or other filtering device must support resolution of domain names to IP addresses.

  2. In the tables below, any destination specified only by domain name must be resolved by the customer if an IP address is required. Many common tools such as dig, drill or nslookup can be used for resolving DNS names to IP addresses. Note that third-party (non-ThousandEyes) DNS mappings may change without warning.

  3. thousandeyes.com domain names are not currently protected by DNSSEC.

  4. Direction assumes rules or ACLs use dynamic/stateful filtering, which permit response packets automatically. If your firewall or filter device uses static packet filters, you must create rules in both directions of the communication.

Network Address Translation

Firewalls or similar devices which use rules or ACLs are typically also capable of performing network address translation (NAT). If your Enterprise Agent is behind a NAT device, then ensure that the necessary NAT rule for your agent exists for the types of tests that the agent will run. ThousandEyes recommends creating static, "one-to-one" NAT rules for the agent as the simplest configuration for proper test function.

Proxy Servers

Some organizations require an Enterprise Agent to use a proxy server for HTTP-based communication. A proxy may be configured with an allowed list of destinations, which are normally specified by domain names (some proxies may require IP addresses or both). For environments which require a proxy, consult your organization's proxy administrator and the articles Installing Enterprise Agents in Proxy Environments and Configuring an Enterprise Agent to Use a Proxy Server.

Agents will still require configuration of rules or ACLs for non-HTTP based communication, which is typically not sent via proxy servers. Most notably, the Network layer data (overview metrics and path visualization) can only be obtained to the proxy but cannot be transmitted through a proxy to the target server. If Network layer data to the target server is desired, then the proxy will need to be bypassed and firewall rules or ACLs configured to allow the Network layer communication directly from the agent to the target.

Rules

Rules are divided into four section: 1) base rules that are required for all agents, 2) rules specific to an agent's installation type, 3) rules required for tests run by an agent, and miscellaneous rules. The latter three categories have multiple sections and subsections. To construct rules for your installation, review the relevant sections and subsections in each category to identify all needed rules.

Use the links in the list below for quick navigation to a specific section of this document.

Rules in each category are cumulative. Add base rules plus the applicable rules for your Enterprise Agent installation type and tests and to obtain the complete ruleset needed for a given agent.

Base Rules

The following communication is required for installation and full functionality of all Enterprise Agents.

Protocol

Port

Destination address and/or name

Notes

TCP, UDP

53

DNS server IP address(es)

Domain Name Service

UDP

123

NTP server domain names or IP addresses

Time synchronization

TCP

443

c1.thousandeyes.com, sc1.thousandeyes.com, ntrav.thousandeyes.com, crashreports.thousandeyes.com, bbot-sentry-proxy.thousandeyes.com, data1.agt.thousandeyes.com, api.thousandeyes.com or 13.248.200.34, 13.248.204.16, 18.144.149.12, 54.177.34.231, 54.193.142.147, 54.219.22.100, 54.241.250.221, 54.241.50.7, 54.241.92.230, 75.2.105.19, 75.2.122.52, 75.2.27.3, 75.2.45.13, 75.2.49.1, 75.2.66.34, 76.223.69.131, 76.223.72.156, 76.223.76.176, 76.223.82.139, 76.223.86.189, 99.83.133.153, 99.83.136.191, 99.83.143.133, 99.83.165.166, 99.83.242.129, 99.83.250.143, 204.236.184.131, 204.236.190.123

ThousandEyes Agent infrastructure

Some organizations may not require rules for DNS and/or NTP servers if both the agent and servers are located inside the organization's network, and thus this communication is not blocked by existing rules or ACLs.

Additionally, ThousandEyes recommends permitting all ICMP error message types inbound to the agent in order to ensure full network functionality. If your firewall is fully stateful/dynamic for all ICMP error response types, then no rules are required. For firewalls which do not dynamically allow ICMP error messages in response to packets sent outbound that encounter the error conditions, we recommend allowing inbound the following:

Protocol

ICMP Types

IPv4

3, 11

IPv6

1-4, 129

Consult your firewall vendor's documentation or contact their technical support to determine whether you need to add rules to allow these ICMP error message responses. Be aware that explicit NAT rules may also be required for the inbound ICMP if the agent is behind a NAT IP address.

Installation Type Rules

Installation types are displayed in the Add New Enterprise Agent dialog of the Enterprise Agents page. Additionally, the "Type" filter of the Enterprise Agents page displays a listing of the types of currently installed (active and deactivated) Enterprise Agents belonging to the current Account Group.

Determine the installation type of your Enterprise Agents and refer the applicable section(s) below. Some installation types fall under more than one section's set of rules (i.e. rules are cumulative, per the infobox above). For example, A ThousandEyes Virtual Appliance requires the rules in the Appliances and Containers section and the rules in the Appliances section within the Appliances and Containers section.

Appliances and Containers

ThousandEyes Appliances and Containers are based on the Ubuntu Linux operating system, and require access to both the generic Ubuntu software package repositories and the ThousandEyes repository to update software automatically. The following rules are required for agents distributed in virtual machine format (Virtual Appliances and Hyper-V Appliances, Cisco- and Juniper-based agents) and for Physical Appliances and Raspberry Pi-based agents which are distributed via ISO image:

Protocol

Port

Destination

Notes

TCP

80

archive.ubuntu.com

Ubuntu Linux package repository

TCP

80

security.ubuntu.com

Ubuntu Linux package repository

TCP

80

archive.canonical.com

Ubuntu Linux package repository

TCP

80 or 443

apt.thousandeyes.com

ThousandEyes APT package repository

Select port 80 or port 443 depending on your organization's security requirements.

The apt.thousandeyes.com repository is located in a content delivery network (CDN), where IP addresses can change without notice. For customers requiring a static IP address for the ThousandEyes APT repository, the aptproxy.thousandeyes.com domain name will always resolve to the same IP addresses. See the ThousandEyes article Static IP Addresses for ThousandEyes Repositories for additional information.

Appliances

ThousandEyes Appliances--which include Virtual Appliances, Physical Appliances, Hyper-V Appliances, and agents installed on Cisco or Raspberry Pi platforms--provide a web-based administration interface, as well as an SSH server for command-line management. The direction of the connections are inbound to the agent (agent IP address is the destination). If web or SSH connections traverse a firewall, the following rules are required:

Protocol

Port

Destination

Notes

TCP

443

Agent IP address

Inbound to the agent

TCP

22

Agent IP address

Inbound to the agent

For more information, see How to set up the Virtual Appliance.

Raspberry Pi

The following rule is required for agents installed on Raspberry Pi 4 hardware:

Protocol

Port

Destination

Notes

TCP

80

ports.ubuntu.com

Ubuntu Linux package repository

Docker containers

The following rule is required only for installation of Docker container-based agents that download images maintained in the Docker registry (as installed with the default docker pull command; see Enterprise Agent Deployment Using Docker)

Protocol

Port

Destination

Notes

TCP

443

hub.docker.com

Docker-based agents (install only)

TCP

443

auth.docker.io

Docker-based agents (install only)

TCP

443

registry.docker.io

Docker-based agents (install only)

TCP

443

production.cloudflare.docker.com

Docker-based agents (install only)

Linux packages

ThousandEyes agent software is supplied by either the ThousandEyes APT repository apt.thousandeyes.com (for Ubuntu), or the ThousandEyes YUM repository yum.thousandeyes.com (For Red Hat, CentOS and Oracle Linux). Agent software has dependencies on software packages provided in common repositories that typically are required for the operating system. Consult your operating system's documentation for the locations of these repositories and construct rules as required.

The apt.thousandeyes.com and yum.thousandeyes.com repositories are located in a content delivery network (CDN), where IP addresses can change without notice. For customers requiring a static IP address for the ThousandEyes APT or YUM repositories, the aptproxy.thousandeyes.com and yumproxy.thousandeyes.com domain names will always resolve to the same IP addresses. See the ThousandEyes article Static IP Addresses for ThousandEyes Repositories for additional information.

For all Linux package installs, if the ThousandEyes BrowserBot package has been installed (implements the Page Load and the Web Transaction test types) and if host-based Linux-based firewall software is employed then following rule is required:

Protocol

Port

Destination

Notes

TCP

8998

127.0.0.1

BrowserBot sandbox (for host-based firewalls only)

Agent processes make internal network connections (i.e. not using the physical network) to the BrowserBot sandbox, which listens on port 8998/TCP of the loopback interface (normally uses IP address 127.0.0.1). Configure the host-based firewall to allow connections to the loopback IP address on the specified port.

Ubuntu

The following rules are required for Ubuntu Linux package installations:

Protocol

Port

Destination

Notes

TCP

80 or 443

apt.thousandeyes.com

ThousandEyes APT package repository

Select port 80 or port 443 depending on your organization's security requirements.

Red Hat

The following rules are required for Red Hat Enterprise Linux, CentOS and Oracle Linux package installations:

Protocol

Port

Destination

Notes

TCP

80 or 443

yum.thousandeyes.com

ThousandEyes YUM package repository

Select port 80 or port 443 depending on your organization's security requirements.

IaaS

ThousandEyes provides IaaS Enterprise Agent templates in Amazon Web Services (AWS). Amazon provides software repositories within the AWS region that the Enterprise Agent EC2 instance requires, and access is available by default from the VPC if the deployment follows the instructions provided in the article IaaS Enterprise Agent Deployment - Amazon AWS. Additionally, access to the ThousandEyes package repository should also be available by default.

However, if a firewall or similar device blocks outbound connections from the EC2 instance, then add the following rule:

Protocol

Port

Destination

Notes

TCP

80 or 443

apt.thousandeyes.com

ThousandEyes APT package repository

Select port 80 or port 443 depending on your organization's security requirements.

Additionally, permit access to the AWS regional repository and any others listed in the /etc/apt/sources.list file of the Enterprise Agent.

For example, an agent in AWS region us-west-1 would require the following rules to access the region's repository, us-west-1.ec2.archive.ubuntu.com and to security.ubuntu.com, per the sources.list file.

Protocol

Port

Destination

Notes

TCP

80

us-west-1.ec2.archive.ubuntu.com

AWS regional Ubuntu Linux package repository

TCP

80

security.ubuntu.com

Ubuntu Linux package repository

Substitute the appropriate destinations from your configuration.

Test Rules

The protocol, port, and destination of rules to permit test traffic will depend on the type of test created and the target (destination) of the test. Normally, the direction for test rules is outbound from the agent. However, for agent-to-agent tests and RTP Stream tests, agents are both sources and target of the test, so the direction for test rules is both outbound and inbound, as indicated below.

The sections below use the default ports for the test types. For example, a Web layer test will need outbound access on TCP port 80 and/or 443 by default. If a test is configured with a non-default port number then the rule must use that port number instead of the default.

Additionally, most non-Network layer tests include Network measurements via the Perform network measurements setting (configured by default, under the Advanced Settings tab of the test configuration). When using Perform network measurements, additional rules may be required for those measurements, in addition to any rules based on the test type. Use the instructions in the Agent-to-Server section below to add any needed rule for your test's network measurements, treating the Protocol field on your test's Advanced Settings tab as the Protocol field in the agent-to-server test.

Similarly, if the Perform network measurements setting includes Collect BGP data then use the instructions in the Routing Layer section below to add any needed rules for your test's network measurements.

Routing Layer

The Routing Layer contains the BGP test type which provides the BGP Route Visualization. The BGP Route Visualization is also part of the Perform network measurements setting of other test types. BGP data is supplied by public BGP Monitors which report data to ThousandEyes, or customers may create Private BGP Monitors using their own BGP-enabled devices to peer with ThousandEyes. If a Private BGP Monitor is used for a BGP test or as part of other tests' Network metrics, and that Private BGP Monitor traverses a firewall, then a firewall rule or ACL may be required.

Private BGP Monitors are independent of Enterprise Agents, but can provide data for tests run by Enterprise Agents, so are included in this article. If your organization uses Private BGP Monitors, review the article Inside-Out BGP Visibility for additional information.

BGP

Private BGP Monitors peer with a router in the ThousandEyes infrastructure. When the Private BGP Monitor is first configured, customers are sent the domain name of the ThousandEyes peer, along with peering instructions. If a Private BGP Monitor traverses a firewall, then the following firewall rule or ACL may be required.

Protocol

Port

Destination

Notes

TCP

179

ThousandEyes peer

Obtain peer's domain name from configuration instructions email

The source is the customer's BGP-speaking device, not an Enterprise Agent. The destination information can be obtained from the email that ThousandEyes sends after a Private BGP Monitor is requested. ThousandEyes emails configuration information to the requestor, including the peer's domain name (for example bgpc1.thousandeyes.com) or IP address.

Network Layer

Network layer tests permit a choice of protocol (TCP, UDP, and/or ICMP depending on the test type). Create rules with the protocol configured in the test's Protocol setting.

Agent-to-Server

Agent-to-server tests default to TCP as the protocol and 80 as the port. If a different port number is used in the test, use that port number in the rule.

Protocol

Port

Destination

Notes

TCP

80

test target

If test's Protocol setting is TCP

Alternatively, if ICMP is selected in the Protocol field on the Basic Configuration tab of the test settings then use the following rule.

Protocol

ICMP service

Destination

Notes

ICMP

ping

test target

Overview metrics (Loss, Latency and Jitter) and Path Visualization

Note that ICMP does not use port numbers, but rather Types and Codes. For the Overview metrics rule, the outbound ICMP code and type uses Type 8, Code 0 (Echo Request) and the return/inbound ICMP uses code 0, type 0 (Echo Reply). Many firewalls refer to this combination as the "ping" service or object.

For the Path Visualization, the outbound ICMP packets use type 8, code 0 (Echo Request) and the returning inbound ICMP packets use both type 0, code 0 (Echo Reply) and type 11, code 0 (Time to Live exceeded in Transit). Many firewalls refer to this combination as the "traceroute" service or object. Normally, a separate rule for traceroute is not needed because stateful firewalls associate the outbound packets (whether TCP, UDP or ICMP) with the inbound ICMP type 11 packets generated by the outbound packets. However, if a firewall cannot make this association, a second rule to allow the type 11 packets inbound to the agent may be required. Such a rule may be similar to the above ICMP rule but with the "traceroute" object, or may require a rule for ICMP type 11 to the agent as destination, from any source.

Note also that "many-to-one" network address translation may not make the association between outbound packets and incoming ICMP type 11 packets, and will block the type 11 packets. A one-to-one NAT rule will need to be configured to allow the inbound ICMP type 11 packets to be correctly translated.

Agent-to-Agent

Agent-to-agent tests default to TCP as the protocol and 49153 as the port. Alternatively, if UDP is selected in the Protocol field on the Basic Configuration tab of the test settings then use UDP. If a different port number is used in the test, use that port number in the rule.

Protocol

Port

Destination

Notes

TCP or UDP

49153

test target

Use TCP or UDP per test's Protocol setting

With agent-to-agent tests, if one or both agents is behind a network address translation device, the NAT traversal feature may be required, particularly if the NAT is not a one-to-one NAT. If required, the Behind a NAT box must be checked on the Enterprise Agent's Settings page.

NAT traversal requires communication to the ThousandEyes NAT traversal service, which may require an additional rule. Use the first rule for TCP-based tests; the second for UDP-based tests.

Protocol

Port

Destination

Notes

TCP

9119 and 9120

ntrav.thousandeyes.com

TCP-based agent-to-agent tests

TCP and UDP

9119 and 9120

ntrav.thousandeyes.com

UDP-based agent-to-agent tests

If a many-to-one type of NAT is used, then the NAT device should meet the criteria in the article NAT Traversal for Agent-to-Agent Tests for agent-to-agent tests.

DNS Layer

DNS Layer tests all use a destination port of 53. The port is not user-configurable. Additionally, DNS Layer tests use only one transport protocol, either UDP or TCP. Truncated responses will never result in a test switching from UDP to TCP.

DNS Server

DNS Server tests default to UDP as the protocol, as specified in the Transport field on the Advanced Settings tab of the test settings. Alternatively, TCP may be used.

Protocol

Port

Destination

Notes

UDP or TCP

53

test target

Use UDP or TCP per test's Transport setting

DNS Trace

DNS Trace tests default to UDP as the protocol, as specified in the Transport field on the Advanced Settings tab of the test settings. Alternatively, TCP may be used.

Protocol

Port

Destination

Notes

UDP or TCP

53

all destinations

Use UDP or TCP per test's Transport setting

Normally, the test must have access to all destinations in order to access all of the servers required to perform iterative queries to authoritative nameservers in the DNS hierarchy, starting from the root nameservers.

DNSSEC

DNSSEC tests are similar to DNS Trace tests, except that DNSSEC tests always use UDP as the transport protocol.

Protocol

Port

Destination

Notes

UDP

53

all destinations

Normally, the test must have access to all destinations in order to access all of the servers required to perform iterative queries to authoritative nameservers in the DNS hierarchy, starting from the root nameservers.

Web Layer

Web layer tests (other than FTP Server tests) differ from other tests in that the target of the test is potentially (or likely) not the only destination for traffic from the agent. HTTP Server tests can receive HTTP redirects to domains other than the target domain name or IP address. Moreover. Page Load and Transaction tests load entire web pages which typically require connections to many destinations. For this reason, the Destination column in the Page Load and Transaction test section indicates "all destinations". If the domains to which requests are made are known, rules can be created which specify only those domains.

HTTP Server

The HTTP Server test uses port 80 by default if the test target is configured with the http:// scheme and uses port 443 if configured with the https:// scheme. If a non-default port number is used by the target server, use that port number in the rule.

Protocol

Port

Destination

Notes

TCP

80 or 443

test target

HTTP or HTTPS (defaults); test target may redirect to a different destination(s)

Typically, a request using HTTP to port 80 is redirected to the HTTPS service on port 443.

Page Load and Transaction

Normally, for Page Load and Transaction tests (a.k.a. Browser-based tests) allowing the agent to access all destinations using HTTP and HTTPS is the easiest way to configure the ruleset, unless the destinations are well known and few in number.

Protocol

Port

Destination

Notes

TCP

80 and 443

all destinations

HTTP and HTTPS (default ports)

Note that when an agent running browser-based tests is configured to use a proxy server, some amount of HTTP-based communication cannot be proxied. Specifically, HTTP-based downloads of SSL/TLS digital certificates (AIA fetching) and certificate revocation lists (CRLs) as well as the Online Certificate Status Protocol (OCSP) are not currently proxy-aware. Under certain circumstances (OCSP stapling unavailable, sites using EV certificates) a Browser-based test could experience errors if the agent cannot perform these types of communication directly. In this situation, two options exist:

  • Create a firewall rule or ACL which permits HTTP connections (typically using the http:// scheme) from the agent to the site required

  • Create a firewall rule or ACL which responds with a TCP reset to the connection attempts from the agent

Contact ThousandEyes Customer Engineering for additional information.

FTP Server

FTP Server tests can use one of three TCP-based protocols: FTP (Active or Passive modes), FTPS or SFTP. The FTP server test uses the following ports by default:

  • Port 21 if the test target is configured with the ftp:// scheme, and port 20 (inbound to the agent from the target server) if Active mode is configured in the Advanced Settings tab

  • Port 990 if configured with the ftps:// scheme

  • Port 22 if configured with the sftp:// scheme

If a non-default port number is used by the target server, use that port number in the rule.

Protocol

Port

Destination

Notes

TCP

21

test target

FTP command channel

TCP

20

Enterprise Agent

FTP data channel (Active mode only)

TCP

990

test target

FTPS

TCP

22

test target

SFTP

Voice Layer

Voice Layer provide tests for control and data streams of a voice-over-IP (VoIP) call, using SIP and RTP, respectively. The SIP Server test connects to a server, proxy, session border controller (SBC) or gateway, on the customer's premises or in the cloud. The RTP Stream test is performed between two ThousandEyes Agents to assess the quality of voice data given the characteristics of the network path.

SIP Server

SIP Server tests default to TCP as the protocol and 5060 as the port number. Alternatively, if UDP is selected in the Protocol field on the Basic Configuration tab of the test settings then use UDP, or if TLS is selected in the Protocol field then use TCP as the protocol and 5061 as the port number. If a different port number is used in the test, use that port number in the rule.

Protocol

Port

Destination

Notes

TCP or UDP

5060

SIP server/proxy/SBC

SIP Server test

TCP

5061

SIP server/proxy/SBC

SIP Server test over TLS

Select one of the two rules above per your test's configuration.

RTP Stream

The RTP Stream test is performed between two ThousandEyes Agents--similar to an agent-to-agent test. Review the requirements for agent-to-agent test rules. RTP Stream tests default to 49152 as the port number. If a different port number is used in the test, use that port number in the rule.

Protocol

Port

Destination

Notes

UDP

49152

Cloud or Enterprise Agent

RTP Stream test

Miscellaneous

Enterprise Agents may require additional rules for optional configurations, such as Kerberos/Active Directory authentication, proxy server configurations, or the Device Layer.

Kerberos and Active Directory

Enterprise Agents which use Kerberos authentication (which is used by Microsoft's Active Directory) to authenticate HTTP requests to web servers or proxies must be able to reach the Kerberos domain controller (KDC) listed in the configuration's KDC Host field on the Kerberos Settings page. If using a Kerberos configuration for an agent, and if communication to the Kerberos domain controller traverses a firewall, then the following rule is required:

Protocol

Port

Destination

Notes

UDP and TCP

88

KDC

Kerberos/AD authentication

The Kerberos settings default to port number 88. If a different port number is used in the configuration's KDC Port field then use that port number in the rule.

Proxies

Enterprise Agents can be configured to use one or more proxy servers for tests, administrative communications, or both. These configurations may require additional firewall rules or ACLs.

Proxy Servers

An organization's proxy servers may be deployed on the same internal networks as Enterprise Agents, or the proxies may be cloud-based, including SaaS-based proxy solutions. If communication to any configured proxy server traverses a firewall, then the following rule is required:

Protocol

Port

Destination

Notes

TCP

proxy port(s)

proxy server(s)

One or more ports per proxy

A proxy server may use one port number for all connections or may use multiple ports for different protocols--most commonly one port for HTTP connections and a second for HTTPS connections. Review your organization's proxy configuration documentation or contact your proxy server administrators to determine what port(s) are used by all proxies that the agent will use.

PAC file Servers

When a client such as a browser or an Enterprise Agent must use multiple proxy servers (for redundancy, optimal performance or other reasons), the client can be configured to use a proxy auto-configuration (PAC) file to select a proxy to handle each HTTP request. The PAC file must be retrieved from a web server at client start-up. If communication to the PAC file's web server traverses a firewall, then the following rule is required:

Protocol

Port

Destination

Notes

TCP

80 or 443

all destinations

HTTP or HTTPS (default ports)

Select the appropriate port number based on the scheme (http:// or https://) of the PAC file's URL.

Device Layer

ThousandEyes Device Layer feature uses the Simple Network Management Protocol (SNMP) to communicate with networked devices. Agents send SNMP GET requests to networked devices either when configured as the targets of Device Discovery, or after the devices have been discovered. If communication to the targeted device traverses a firewall, then the following rule is required:

Protocol

Port

Destination

Notes

UDP

161

target device

SNMP GET

Additional devices may be discovered without explicitly specifying an IP address in a discovery's Targets field. The discovery can occur even if those devices are blocked from the agent by a firewall, but the agent will not be able to retrieve data. For those discovered devices, a similar rule to the above will be required, using the discovered device:

Protocol

Port

Destination

Notes

UDP

161

discovered device

SNMP GET

Internet Insights

ThousandEyes' Internet Insights feature aggregates data from existing tests of various types. Because no tests are specific to Internet Insights, no firewall rules or ACLs are required to use Internet Insights.