What to Do If te-agent Stops Running Due to a VACUUM Error

The ThousandEyes Enterprise Agent stores results of tests in a local database. When the agent checks in with the ThousandEyes collector, the contents of the database are uploaded to the collector, and the database is purged. In certain circumstances, the local copy of the database can become corrupted, due in large part to disk I/O errors associated with the filesystem storing the database. This phenomenon is documented at https://www.sqlite.org/howtocorrupt.html.

When database corruption occurs the Agent will stop, and an error message will be seen in /var/log/te-agent.log, similar to the following:

2013-02-06 18:30:06.232 INFO  [te] - Agent version 0.12.1 starting.  Setting max core size to ...
2013-02-06 18:30:06.236 FATAL [te] - Unable to set up agent tables: Error executing "VACUUM", err=11. Please contact [email protected] for assistance.
2013-02-06 18:30:06.237 ERROR [te] - Error updating agent status: Error preparing query UPDATE agent_status SET time = datetime('now'), status = 'Unable to set up agent tables: Error executing "VACUUM", err=11. Please contact [email protected] for assistance.': err=1

As a result, the local database will be stopped and the Agent will be unable to operate. The Agent's status will be shown as "Offline" in the Agent Settings page, with a last contacted date of whenever the first VACUUM error occurred.

In order to resolve this, follow the instructions below, per the type of Enterprise Agent (Linux package or Virtual Appliance).

Linux Package

As root, stop the te-agent process, then go to the /var/lib/te-agent/ directory and remove the te-agent.sqlite file, then restart the te-agent process:

Ubuntu 18.04 and 20.04, Red Hat Enterprise Linux 7 and 8, CentOS 7:

systemctl stop te-agent
cd /var/lib/te-agent
rm te-agent.sqlite
systemctl start te-agent

Alternative method when package "initscripts" (RHEL 7, CentOS 7) is installed:

service te-agent stop
cd /var/lib/te-agent
rm te-agent.sqlite
service te-agent start

Virtual Appliance

Log into the Virtual Appliance web console and click on the Advanced Settings tab. Click the Clear Result Cache button and confirm.

Contacting ThousandEyes Support

If steps described above do not resolve the issue or if the issue keeps reoccurring, please contact ThousandEyes Customer Engineering team using chat or [email protected] email address.