Installing Enterprise Agents on Cisco Switches with Docker

This article walks users through the steps to install a ThousandEyes Enterprise Agent on a Cisco Catalyst 9000-series switch with Docker, using the command line. The Enterprise Agent is a signed ThousandEyes Docker image that can be launched using Cisco application hosting.

The agent can also be installed using the Cisco DNA Center.

Overview

To support application hosting capabilities on Cisco Catalyst 9000-series switches, the switch provides hardware resources where applications can reside and execute. Cisco IOS XE reserves dedicated memory and CPU resources for application hosting to provide a separate execution space for user applications, without compromising the integrity and performance of the switch.

The Cisco IOS XE 16.12.1 release introduced native Docker container support on Catalyst 9000-series switches. The ThousandEyes Enterprise Agent leverages this capability to run a Docker container hosted on internal flash storage (if no SSD is available).

Container connectivity is described in the image below. Containers can be connected via the management interface and front panel data ports. The management interface connects to the container interface via the management bridge, and the IP address of the container will be on the same subnet as the management interface. Virtual network interface cards (vNICs) inside containers are seen as standard Ethernet interfaces (eth0, eth1, etc.).

For more information on Cisco application hosting, see Application Hosting.

Requirements

For detailed requirements for installing Enterprise Agents on Cisco Catalyst 9000-series switches, see the Support Matrix.

Installation Steps

ThousandEyes supports configuring multiple interfaces on Cisco Catalyst devices. For more information, see Multi-Interface Support for Cisco Catalyst 9000 Switches.

Downloading the Docker Image

Download the Docker image from the ThousandEyes dashboard and copy it to your Cisco switch using SCP, FTP, TFTP, or USB storage.

If the switch has internet access, download the image directly onto the switch. Download the package from the ThousandEyes downloads site.

  1. Log in to the ThousandEyes platform using a login belonging to the account group that will be associated with the appliance.

  2. Go to Cloud & Enterprise Agents > Agent Settings and click Add New Enterprise Agent.

  3. Download the .tar file with the ThousandEyes appliance for Catalyst 9000-series switches.

  4. Use SCP, FTP, TFTP, or USB storage to copy the signed Docker image to the switch's flash: directory.

    copy scp://thousandeyes@10.100.21.239/thousandeyes-enterprise-agent-4.4.2.cisco.tar flash:

  5. Run a checksum (md5) command to verify that the package transfer was successful. The md5 output should match 14b88bfc3ec75a2ff4414d8f39106a29:

    catalyst#verify /md5 flash:thousandeyes-enterprise-agent-4.4.2.cisco.tar
-----------------------------------------------------------
verify /md5 (flash:thousandeyes-enterprise-agent-4.4.2.cisco.tar) = 14b88bfc3ec75a2ff4414d8f39106a29

Installing the Docker Container

  1. Enable the IOx framework on the switch:

    Enter configuration commands, one per line.  End with CNTL/Z.
catalyst(config)#iox
catalyst(config)#end

  2. Wait until all the services are running:

    catalyst#show iox-service
​IOx Infrastructure Summary:
---------------------------
IOx service (CAF) 1.11.0.5     : Running
IOx service (HA)               : Running
IOx service (IOxman)           : Running
IOx service (Sec storage)      : Not Running
Libvirtd 1.3.4                 : Running
Dockerd 18.03.0                : Running
Application DB Sync Info       : Available
Sync Status                    : Disabled

  3. Run the install command:

    catalyst#app-hosting install appid <app-name> package flash:thousandeyes-enterprise-agent-4.4.2.cisco.tar

    Specify your desired app name and the location of the image file you want to use. In this example, we use thousandeyes_enterprise_agent.

  4. If the image is hosted on an HTTPS server, you can run the following command to download the image:

    catalyst#app-hosting install appid <app-name> package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-4.4.2.cisco.tar

  5. Your application should now be installed. You can check on it by running the following:

    catalyst#sh app-hosting list
App id State
thousandeyes_enterprise_agent DEPLOYED

Configuring the Docker Container

Docker supports both guest IP address assignment and dynamic IP address assignment. You must configure a single virtual network interface card (vNIC) for the appliance that would allow the Layer-2 VLAN routed from the uplink switch and router to be assigned to the container.

Ensure that the Layer-2 VLAN has been passed through from any active physical port and is not the default VLAN used in the switch (usually VLAN 1).

  1. Verify that the front panel data port is running, with Layer-2 VLAN allowed from uplink:

    catalyst(config)#interface GigabitEthernet1/0/13
catalyst(config-if)#description Uplink MGMT
catalyst(config-if)#switchport access vlan 21

  2. Verify that the Layer-2 VLAN is created:

    catalyst(config)#vlan 21

  3. Configure the AppGigabitEthernet port to allow Layer-2 VLAN:

    catalyst(config)#interface AppGigabitEthernet1/0/1
catalyst(config-if)#switchport trunk allowed vlan 21,22,23,24
catalyst(config-if)#switchport mode trunk

  4. Configure the application, either with a static IP or with DHCP IP.

    Configuration with Static IP

    Use a guest IP address to assign a static IP address. In this example, assign 10.100.21.222/24, under VLAN 21 and use Google resolver:

    catalyst(config)#app-hosting appid thousandeyes_enterprise_agent
catalyst(config-app-hosting)#app-vnic AppGigabitEthernet trunk
catalyst(config-config-app-hosting-trunk)#vlan 21 guest-interface 0
catalyst(config-config-app-hosting-vlan-access-ip)#guest-ipaddress 10.100.21.222 netmask 255.255.255.0
catalyst(config-config-app-hosting-vlan-access-ip)#exit
catalyst(config-config-app-hosting-trunk)#exit
catalyst(config-app-hosting)#app-default-gateway 10.100.21.1 guest-interface 0
catalyst(config-app-hosting)#name-server0 8.8.8.8
catalyst(config-app-hosting)#name-server1 8.8.4.4

    Next, set up the required Docker run options to specify account token. If you want to specify a hostname other than the switch's name, do this here as well:

    catalyst(config-app-hosting)#app-resource docker
catalyst(config-app-hosting-docker)#prepend-pkg-opts
catalyst(config-app-hosting-docker)#run-opts 1 "-e TEAGENT_ACCOUNT_TOKEN=<Token>"
catalyst(config-app-hosting-docker)#run-opts 2 "--hostname Cisco-Docker"
catalyst(config-app-hosting-docker)#exit
catalyst(config-app-hosting)#start
catalyst(config-app-hosting)#end

    Configuration with DHCP IP

    Make sure the DHCP server is running on the layer-2 VLAN. In this case, assign a DHCP address under VLAN 21 and use Google resolver:

    catalyst(config)#app-hosting appid thousandeyes_enterprise_agent
catalyst(config-app-hosting)#app-vnic AppGigabitEthernet trunk
catalyst(config-config-app-hosting-trunk)#vlan21 guest-interface 0

    Next, set up the required Docker run options to specify the account token. If you want to specify a hostname other than the switch's name, do this here as well:

    catalyst(config-config-app-hosting-vlan-access-ip)#app-resource docker
catalyst(config-app-hosting-docker)#prepend-pkg-opts
catalyst(config-app-hosting-docker)#run-opts 1 "-e TEAGENT_ACCOUNT_TOKEN=<Token>"
catalyst(config-app-hosting-docker)#run-opts 2 "--hostname Cisco-Docker"
catalyst(config-app-hosting-docker)#name-server0 8.8.8.8
catalyst(config-app-hosting-docker)#exit
catalyst(config-app-hosting)#start
catalyst(config-app-hosting)#end

    For a full list of the Docker configuration options, see Docker Agent Configuration Options.

  5. Use wr mem to ensure that your configuration changes have persisted across reboots:

    catalyst#wr mem
Building configuration...
[OK]

Verifying That the Docker Container Is Running

With the (config-app-hosting)#start command, the Docker container should have been started and should be running.

  1. Verify this by running the following:

    catalyst# sh app-hosting list
App id                                   State
---------------------------------------------------------
thousandeyes_enterprise_agent            RUNNING

  2. Verify the Docker container’s details:

    catalyst#show app-hosting detail appid thousandeyes_enterprise_agent
App id                 : thousandeyes_enterprise_agent
Owner                  : iox
State                  : RUNNING
Application
   Type                 : docker
   Name                 : ThousandEyes Enterprise Agent
   Version              : 4.4.2
   Description          : 
   Author               : ThousandEyes <support@thousandeyes.com>
   Path                 : flash:thousandeyes-enterprise-agent-4.4.2.cisco.tar
   URL Path             : 
Activated profile name : custom

Resource reservation
  Memory               : 500 MB
  Disk                 : 1 MB
  CPU                  : 1850 units
  VCPU                 : 1
Attached devices
  Type              Name               Alias
---------------------------------------------
  serial/shell     iox_console_shell   serial0
  serial/aux       iox_console_aux     serial1
  serial/syslog    iox_syslog          serial2
  serial/trace     iox_trace           serial3

Network interfaces
  ---------------------------------------
eth0:
  MAC address         : 52:54:dd:d:38:3d
  Network name        : mgmt-bridge-v21
Docker
------
Run-time information
  Command              :
  Entry-point          : /sbin/my_init
  Run options in use   : -e TEAGENT_ACCOUNT_TOKEN=TOKEN_NOT_SET
--hostname=$(SYSTEM_NAME) --cap-add=NET_ADMIN --mount
type=tmpfs,destination=/var/log/agent,tmpfs-size=140m --mount
type=tmpfs,destination=/var/lib/te-agent/data,tmpfs-size=200m -v
$(APP_DATA)/data:/var/lib/te-agent -e TEAGENT_PROXY_TYPE=DIRECT -e
TEAGENT_PROXY_LOCATION= -e TEAGENT_PROXY_USER= -e
TEAGENT_PROXY_AUTH_TYPE= -e TEAGENT_PROXY_PASS= -e
TEAGENT_PROXY_BYPASS_LIST= -e TEAGENT_KDC_USER= -e TEAGENT_KDC_PASS=
-e TEAGENT_KDC_REALM= -e TEAGENT_KDC_HOST= -e TEAGENT_KDC_PORT=88 -e
TEAGENT_KERBEROS_WHITELIST= -e TEAGENT_KERBEROS_RDNS=1 -e PROXY_APT=
-e APT_PROXY_USER= -e APT_PROXY_PASS= -e APT_PROXY_LOCATION= -e
TEAGENT_AUTO_UPDATES=1 -e
TEAGENT_ACCOUNT_TOKEN=nfhjzm8e8ikg07d4n31wcsws9bakcloh --hostname
Cisco-Docker

  Package run options  : -e TEAGENT_ACCOUNT_TOKEN=TOKEN_NOT_SET
--hostname=$(SYSTEM_NAME) --cap-add=NET_ADMIN --mount
type=tmpfs,destination=/var/log/agent,tmpfs-size=140m --mount
type=tmpfs,destination=/var/lib/te-agent/data,tmpfs-size=200m -v
$(APP_DATA)/data:/var/lib/te-agent -e TEAGENT_PROXY_TYPE=DIRECT -e
TEAGENT_PROXY_LOCATION= -e TEAGENT_PROXY_USER= -e
TEAGENT_PROXY_AUTH_TYPE= -e TEAGENT_PROXY_PASS= -e
TEAGENT_PROXY_BYPASS_LIST= -e TEAGENT_KDC_USER= -e TEAGENT_KDC_PASS=
-e TEAGENT_KDC_REALM= -e TEAGENT_KDC_HOST= -e TEAGENT_KDC_PORT=88 -e
TEAGENT_KERBEROS_WHITELIST= -e TEAGENT_KERBEROS_RDNS=1 -e PROXY_APT=
-e APT_PROXY_USER= -e APT_PROXY_PASS= -e APT_PROXY_LOCATION= -e
TEAGENT_AUTO_UPDATES=1

Application health information
  Status               : 0
  Last probe error     :
  Last probe output    :

  3. In the ThousandEyes platform, go to Cloud & Enterprise Agents > Agent Settings and verify the Docker container’s IP address:

Assigning the Agent to Tests

Now that you have installed, configured, and started your Docker-based agent, you can create tests and assign them to be run by your new agent. For instructions, see Getting Started with Tests.

Modify the Docker Container

  1. Stop the application:

    catalyst# app-hosting stop appid thousandeyes_enterprise_agent
thousandeyes_enterprise_agent stopped successfully
Current state is: STOPPED

  2. De-activate the application:

    catalyst# app-hosting deactivate appid thousandeyes_enterprise_agent
thousandeyes_enterprise_agent deactivated successfully
Current state is: DEPLOYED

  3. Modify the Docker options, and exit three times:

    catalyst(config)#app-hosting appid thousandeyes_enterprise_agent
catalyst(config-app-hosting)#app-resource docker
catalyst(config-app-hosting-docker)#prepend-pkg-opts
catalyst(config-app-hosting-docker)#<run-opts command>
catalyst(config-app-hosting-docker)#exit
catalyst(config-app-hosting)#exit
catalyst(config)#exit

  4. Reactivate the application, and confirm that it’s activated:

    catalyst# app-hosting activate appid thousandeyes_enterprise_agent
thousandeyes_enterprise_agent activated successfully
Current state is: ACTIVATED

  5. Start the application, and confirm that it is running:

    catalyst# app-hosting start appid thousandeyes_enterprise_agent
thousandeyes_enterprise_agent started successfully
Current state is: RUNNING

Multi-Interface Support for Cisco Catalyst 9000 Switches

ThousandEyes supports configuring multiple interfaces on Cisco Catalyst devices, allowing Cisco Catalyst Enterprise Agents to access multiple virtual networks with the same Enterprise Agent. Once configured, users can select which interface to use for a test from the agent selection UI.

For more information on interface selection, see Enterprise Agent Interface Selection.

Supported Devices

The following devices are supported for configuring multiple interfaces:

  • Cisco Catalyst 9300

  • Cisco Catalyst 9400

For more information on supported Cisco devices, see the Support Matrix.

Prerequisites

The app-hosting container on the Cisco Switch must be using image version 4.3.0 or later for multi-interface support.

For more detailed requirements for installing Enterprise Agents on Cisco Catalyst 9000-series switches, see the Support Matrix.

Configuration

To avoid any ARP issues, ThousandEyes suggests limiting your environment to one guest IP address per VLAN in the app-vnic configuration.

To configure multiple interfaces, you need to configure one or more additional guest-interfaces and associate them with the relevant networks or VLANs by using the app-default-gateway configuration command. Once configured, you can run tests over the additional interfaces by specifying the default gateways for each of the networks associated with these interfaces, using environment variables in the container to specify the default gateway address, guest-ipaddress, and VLAN ID. The environment variables should follow the following naming convention, where X is any value in the range of 0-7, and corresponds to the number of the guest interface in the container configuration:

TEAGENT_DEF_IPV4_GW_ETH<X>

  1. If reconfiguring an existing container, first stop the application:

    catalyst# app-hosting stop appid thousandeyes_enterprise_agent
thousandeyes_enterprise_agent stopped successfully
Current state is: STOPPED

  2. De-activate the application:

    catalyst# app-hosting deactivate appid thousandeyes_enterprise_agent
thousandeyes_enterprise_agent deactivated successfully
Current state is: DEPLOYED

  3. Modify the container:

    app-hosting appid cat9k402
   app-vnic AppGigabitEthernet trunk
      vlan 21 guest-interface 0
         guest-ipaddress 10.100.21.65 netmask 255.255.255.0
      vlan 22 guest-interface 1
         guest-ipaddress 10.100.22.65 netmask 255.255.255.0
      vlan 23 guest-interface 2
         guest-ipaddress 10.100.23.65 netmask 255.255.255.0
      vlan 24 guest-interface 3
         guest-ipaddress 10.100.24.65 netmask 255.255.255.0
         
   app-default-gateway 10.100.21.1 guest-interface 0
app-resource docker
   prepend-pkg-opts
      run-opts 1 "-e TEAGENT_ACCOUNT_TOKEN={}"
      run-opts 2 "--hostname cat9k-multi"
      run-opts 3 "-e TEAGENT_DEF_IPV4_GW_ETH1=10.100.22.10"
      run-opts 4 "-e TEAGENT_DEF_IPV4_GW_ETH2=10.100.23.10"
      run-opts 5 "-e TEAGENT_DEF_IPV4_GW_ETH3=10.100.24.10"
      name-server0 8.8.8.8
      name-server1 10.100.50.102

  4. Exit three times to completely exit out of config mode.

  5. Use wr mem to ensure the changes are persistent across reboots.

Once the configuration has been saved, reactivate and restart the container to apply the app-hosting configuration changes.

The image below show the configured routing table in the ThousandEyes web app, in Agent Settings > Selected Agent > System Information > Routing Table:

Limitations

  • This process is only supported via app-hosting CLI, not via DNAC or vManage.

  • Browserbot related tests (page load and transaction) are not supported.

  • DNS tests are not supported. DNS requests will continue to be sent via default route and source address.

  • Agent to agent tests are not supported, as there is no interface selection for the return path. The response will continue to use the default route.

Frequently Asked Questions

What is the expected NTP behavior for a Catalyst 9000 series deployed Enterprise agent?

The enterprise agent on a Catalyst 9000 series switch uses the host system kernel clock. It also sends packets to pool.ntp.org to determine any clock offset. It does not try to adjust the host or container clock but will adjust measurement timestamps based on the clock offset.

Can the default external NTP source (pool.ntp.org) be changed to a customer's internal NTP source?

No. The agent uses pool.ntp.org to determine clock offset by default; this is currently not configurable.

What happens if the primary switch in my HA mode stack fails?

When a Cat9k switch is deployed in HA mode (stacked), for the first 30 minutes, if the primary switch in the stack fails, and a secondary switch takes over, a new agent will be brought up, and the original agent on the failed switch will go offline. After the first 30 minutes, there will be seamless agent failover that preserves agent identity.

How do I connect to the agent shell for Cisco agents?

To access the agent shell of a Cisco Enterprise Agent that is actively running, use the following command:

catalyst#app-hosting connect appid {application name} session
#

Once inside the agent shell, you can refer to the agent log for any further troubleshooting:

# tail /var/log/agent/te-agent.log

If connection or DNS resolution errors are found in the log file, your agent cannot connect to the ThousandEyes platform. Check your app-vnic configuration and make sure the agent IP can reach the internet.

For more information on configuration options, see Docker Agent Config Options.

Can I use ThousandEyes troubleshooting utilities?

From Agent 4.0.2 onwards, te-agent-utils are pre-installed on Cisco Enterprise Agents. For more information on the available utilities, see CLI Network Troubleshooting Utilities.

What are the default trusted default root certificates used by the Enterprise Agent Docker container when communicating with ThousandEyes services?

  • issuer=O = Cisco, CN = Cisco Licensing Root CA

  • issuer=O = Cisco, CN = Cisco Basic Assurance Root CA 2099

  • issuer=O = Cisco, CN = Cisco ECC Root CA

  • issuer=O = Cisco Systems, CN = Cisco Root CA 2048

  • issuer=O = Cisco, CN = Cisco Root CA 2099

  • issuer=O = Cisco, CN = Cisco Root CA M1

  • issuer=O = Cisco, CN = Cisco Root CA M2

  • issuer=C = US, O = Cisco Systems, CN = Cisco RXC-R2

  • issuer=C = US, O = Amazon, CN = Amazon Root CA 1

  • issuer=C = US, O = Amazon, CN = Amazon Root CA 2

  • issuer=C = US, O = Amazon, CN = Amazon Root CA 3

  • issuer=C = US, O = Amazon, CN = Amazon Root CA 4

  • issuer=C = NO, O = Buypass AS-983163327, CN = Buypass Class 2 Root CA

  • issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA

  • issuer=C = US, O = Internet Security Research Group, CN = ISRG Root X1

  • issuer=C = US, O = IdenTrust, CN = IdenTrust Commercial Root CA 1

  • issuer=C = BM, O = QuoVadis Limited, CN = QuoVadis Root CA 2

  • issuer=C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust ECC Certification Authority

  • issuer=C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority

  • issuer=C = US, O = Google Trust Services LLC, CN = GTS Root R1

  • issuer=C = US, O = Google Trust Services LLC, CN = GTS Root R2

  • issuer=C = US, O = Google Trust Services LLC, CN = GTS Root R3

  • issuer=C = US, O = Google Trust Services LLC, CN = GTS Root R4

How do I install CA certificates on Cisco devices?

For CA certificate installation instructions, see Installing CA Certificates on Enterprise Agents.

For multi-interface support, which interface is used for the agent default registration? Can I change that?

eth0 is used, and is specified in the configuration as 'guest-interface 0". It is possible to change the interface used by using the app-default-gateway config command to set the default route in the container.

For multi-interface support, is there a limitation on the number of configurable interfaces?

ThousandEyes supports using up to 8 interfaces on Catalyst 9300 and 9400 switches.

PreviousInstalling Enterprise Agents on Cisco Nexus Switches with GuestshellNextInstalling Enterprise Agents on Cisco Routers with Docker

Last updated