Using Webhooks (Sample Server Code Included)

ThousandEyes Documentation
Search…
ThousandEyes Documentation
What's New
Changelog
Archived Release Notes
Using Webhooks (Sample Server Code Included)
In this article, we'll describe the configuration of alerts to use webhooks. Additionally, for those unfamiliar with webhooks or who will benefit from constructing a quick example, we provide information to set up a test web service using Heroku. For reference, we've also included example POST body data at the end of this article.
Configuring Webhook Notifications
To configure notifications to a webhook server, you'll need to define a webhook in the Alert Rules interface (Alerts > Alert Rules). Expand an alert rule and select the Notifications tab. Under the Webhooks section, click the Configure Webhooks link. If you already have configured webhooks, click the Edit webhooks link:
Click Add New Webhook, then complete the New Webhook dialog: 
 Enter the Name of your Webhook, and the target URL (note that HTTP redirects are not accepted, so be sure to provide the final URL). Configure required authentication if required, and click Test to check your configuration: 
 NOTE: Encoded HTML URLs are currently not supported for webhook configuration. Enter the URL in decoded form. If the test succeeds, a Webhook test completed successfully message displays: 
 Once you have tested your webhook server successfully, click Add New Webhook, and select your webhook from the list of configured webhooks. You can add any number of custom, static key-value pairs to the webhook payload. These custom payloads can be used on any Cloud or Enterprise Agent alert type and are specific to the selected alert rule.
To add these elements, navigate to an individual alert rule, select a webhook, and click Add Custom Payload (1). A key-value pair (2) field appears that allows you to enter any static key-value pair. You can add or remove these payloads using the - and + buttons (3).
​
 Additionally, this change adds two new fields to the webhook payload: testLabels, which includes test label name and ID, as well as testTargetsDescription, which includes the description of the test target as defined in the test configuration.
Webhook Authentication
The webhook interface supports the use of a token, OAuth, and basic HTTP authentication. This is an optional configuration step and can be configured for a webhook service if the target server requires authentication.
Basic
A Username and corresponding Password are necessary to configure Basic HTTP Authentication: 
​
Token
The Token Auth Type uses a Bearer Token to authenticate with the webhook server: 
​
OAuth
Webhook Server-Side Configuration
Create a new OAuth endpoint for ThousandEyes Alerts with https://app.thousandeyes.com/webhooks-oauth-callback/ as redirect/call back URL. Grab the newly created Client ID and Authorization Endpoint URL (Auth URL) for the webhook server to configure ThousandEyes-side authentication. Below is an example Client OAuth configuration from ServiceNow: 
 NOTICE: Ensure that the token has a sufficient lifespan to avoid manual regeneration on timeout.
ThousandEyes-Side Configuration
Configure the webhook Name and URL as explained in the Configuring Webhook Notifications section. Click OAuth in Auth Type. Fill out the Auth URL and Client ID according to the Webhook Server-Side Configuration section above. Click Get Token: 
 A new tab opens, requesting consent to authorize ThousandEyes. Below is a snippet from ServiceNow application that we are using in this guide as an example: 
 Once the authorization is complete, a Token retrieved successfully message is displayed, as shown in the following figure: 
 Now click Test to verify the operability of your new webhook: 
​
Setting Up a Webhook Server
If you visit Heroku, you can deploy a free web application to demonstrate the behavior. We'll deploy a simple web service that accepts posts in JSON format, to show what requests from ThousandEyes look like. If you're new to either node.js, or to Heroku, it's worth running through their instructional tutorials to understand how the deployment process works. Follow these instructions to deploy a simple version of your web service.
1.Install the Heroku command line interface (CLI) Go to https://devcenter.heroku.com/articles/getting-started-with-nodejs#set-up, and follow the instructions to download the Heroku CLI for your operating system.
2.Clone the simple-webhook-server repository and move to the directory created.
1
$ git clone https://github.com/thousandeyes/simple-webhook-server.git
2
Cloning into 'simple-webhook-server'...
3
remote: Counting objects: 26, done.
4
remote: Total 26 (delta 0), reused 0 (delta 0), pack-reused 26
5
Unpacking objects: 100% (26/26), done.
6
$ cd simple-webhook-server
Copied!
3.Create a Heroku remote in your git repository.
1
$ heroku create
2
Creating app... done, vast-shore-1766
3
https://vast-shore-1766.herokuapp.com/ | https://git.heroku.com/vast-shore-1766.git
Copied!
4.Push your content to the Heroku remote. This will build the application.
1
$ git push heroku master
2
Counting objects: 12, done.
3
Delta compression using up to 4 threads.
4
Compressing objects: 100% (9/9), done.
5
Writing objects: 100% (12/12), 3.31 KiB | 0 bytes/s, done.
6
Total 12 (delta 0), reused 0 (delta 0)
7
remote: Compressing source files... done.
8
remote: Building source:
9
remote:
10
remote: -----> Node.js app detected
11
remote:
12
remote:        PRO TIP: Specify a node version in package.json
13
remote:        See https://devcenter.heroku.com/articles/nodejs-support
14
remote:
15
remote: -----> Defaulting to latest stable node: 0.10.33
16
remote: -----> Downloading and installing node
17
remote: -----> Exporting config vars to environment
18
remote: -----> Installing dependencies
19
remote:        npm WARN package.json [email protected] No repository field.
20
remote:        [email protected] node_modules/body-parser
21
remote:        ├── [email protected]
22
remote:        ├── [email protected] ([email protected], [email protected])
23
remote:        └── [email protected] ([email protected])
24
remote:
25
remote:        [email protected] node_modules/express
26
remote:        ├── [email protected]
27
remote:        ├── [email protected]
28
remote:        ├── [email protected]
29
remote:        ├── [email protected]
30
remote:        ├── [email protected]
31
remote:        ├── [email protected]
32
remote:        ├── [email protected]
33
remote:        ├── [email protected]
34
remote:        ├── [email protected]
35
remote:        ├── [email protected]
36
remote:        ├── [email protected]
37
remote:        ├── [email protected]
38
remote:        ├── [email protected]
39
remote:        ├── [email protected]
40
remote:        ├── [email protected]
41
remote:        ├── [email protected]
42
remote:        ├── [email protected]
43
remote:        ├── [email protected] ([email protected])
44
remote:        ├── [email protected] ([email protected], [email protected])
45
remote:        ├── [email protected] ([email protected])
46
remote:        ├── [email protected] ([email protected])
47
remote:        ├── [email protected] ([email protected], [email protected], [email protected])
48
remote:        ├── [email protected] ([email protected], [email protected])
49
remote:        └── [email protected] ([email protected])
50
remote: -----> Caching node_modules directory for future builds
51
remote: -----> Cleaning up node-gyp and npm artifacts
52
remote: -----> No Procfile found; Adding npm start to new Procfile
53
remote: -----> Building runtime environment
54
remote: -----> Discovering process types
55
remote:        Procfile declares types -> web
56
remote:
57
remote: -----> Compressing... done, 5.7MB
58
remote: -----> Launching... done, v3
59
remote:        https://vast-shore-1766.herokuapp.com/ deployed to Heroku
60
remote:
61
remote: Verifying deploy... done.
62
To https://git.heroku.com/vast-shore-1766.git
63
 * [new branch]      master -> master
Copied!
5.Open the site to confirm the target location. This opens a web browser in the context of your Heroku application.
1
$ heroku open
Copied!
6.Open a console to review the logs of the target server. In this window, you'll see POST requests hitting your server.
1
$ heroku logs --tail
2
2014-11-11T20:33:42.767724+00:00 heroku[api]: Deploy e5d1a07 by [email protected]
3
2014-11-11T20:33:42.767753+00:00 heroku[api]: Release v15 created by [email protected]
4
2014-11-11T20:33:43.724332+00:00 heroku[web.1]: State changed from up to starting
5
2014-11-11T20:33:46.814245+00:00 heroku[web.1]: Starting process with command `node webhook-server.js`
6
2014-11-11T20:33:50.439787+00:00 heroku[web.1]: State changed from starting to up
7
2014-11-11T20:33:50.347118+00:00 app[web.1]: Webhook Server started... port: 30490
8
2014-11-11T20:33:59.586785+00:00 heroku[router]: at=info method=GET path="/" host=sleepy-mesa-2108.herokuapp.com request_id=228650f8-d810-477e-858b-b61235766260 fwd="54.90.223.146" dyno=web.1 connect=2ms service=10ms status=200 bytes=266
Copied!
Webhook POST Content
Webhooks use events to signify activity. Our system is configured today to support the following types of events:
WEBHOOK_TEST - sent when a user clicks the Test Webhook link in ThousandEyes webhook configuration.
ALERT_NOTIFICATION_TRIGGER - sent when an alert is triggered
ALERT_NOTIFICATION_CLEAR - sent when an alert is cleared, the alert rule is deleted, or the alert rule is removed from a test while in an active alerting status.
AGENT_ALERT_NOTIFICATION_TRIGGER - sent when an agent notification triggers - may be related to clock offset, version number, or agent online status
AGENT_ALERT_NOTIFICATION_CLEAR - sent when the agent notification clears Note: All eventIds are unique. To correlate two events (for example, an alert trigger and clear,) reference the alertId, which is unique per generated alert.
Sample Webhook Event Payloads
Each event type shown below contains a sample JSON payload, which is posted to the target server. The Alert notification data corresponds to alerting data from the ThousandEyes API, wrapped in an event wrapper. See the ThousandEyes Developer site for more information on data returned via the API.
WEBHOOK_TEST
1
{
2
   "eventType":"WEBHOOK_TEST",
3
   "eventId":"1415733887743"
4
}
Copied!
ALERT_NOTIFICATION_TRIGGER
1
{
2
   "eventType":"ALERT_NOTIFICATION_TRIGGER",
3
   "eventId":"49-35541",
4
   "alert":{
5
      "ruleId":173,
6
      "alertId":35541,
7
      "testId":5370,
8
      "testName":"http://www.thousandeyes111.com",
9
      "testLabels":[
10
         {
11
            "id":78399,
12
            "name": "AI Sample Label"
13
         }
14
      ]
15
      "ruleExpression":"Page Load is incomplete",
16
      "dateStart":"2014-08-24 01:01:47",
17
      "violationCount":2,
18
      "ruleName":"Default Page Load Alert Rule",
19
      "agents":[
20
         {
21
            "dateStart":"2014-08-24 01:01:47",
22
            "active":1,
23
            "metricsAtStart":"Page load: Incomplete",
24
            "metricsAtEnd":"Page Load: Incomplete",
25
            "agentId":108,
26
            "agentName":"Boston, MA",
27
            "locationName":"Boston, MA",
28
            "permalink":"https://app.thousandeyes.com/alerts/list?__a=11&alertId=35541&agentId=108"
29
         },
30
         {
31
            "dateStart":"2014-08-24 01:00:51",
32
            "active":1,
33
            "metricsAtStart":"Page load: Incomplete",
34
            "metricsAtEnd":"Page Load: Incomplete",
35
            "agentId":107,
36
            "agentName":"Vancouver, Canada",
37
            "locationName":"Vancouver, Canada",
38
            "permalink":"https://app.thousandeyes.com/alerts/list?__a=11&alertId=35541&agentId=107"
39
         }
40
      ],
41
      "testTargetsDescription": [
42
         "http://google.com"
43
      ],
44
      "type":"Page Load",
45
      "permalink":"https://app.thousandeyes.com/alerts/list?__a=11&alertId=35541",
46
      "apiLinks":[
47
         {
48
            "rel":"related",
49
            "href":"http://api.thousandeyes.com/tests/5370"
50
         },
51
         {
52
            "rel":"data",
53
            "href":"http://api.thousandeyes.com/web/page-load/5370"
54
         }
55
      ]
56
   }
57
}
Copied!
ALERT_NOTIFICATION_CLEAR
1
{
2
   "eventType":"ALERT_NOTIFICATION_CLEAR",
3
   "eventId":"49-35546",
4
   "alert":{
5
      "ruleId":173,
6
      "alertId":35541,
7
      "testId":5370,
8
      "testName":"http://www.thousandeyes111.com",
9
      "testLabels":[
10
         {
11
            "id":78399,
12
            "name": "AI Sample Label"
13
         }
14
      ]
15
      "ruleExpression":"Page Load is incomplete",
16
      "dateStart":"2014-08-24 01:01:47",
17
      "dateEnd":"2014-08-24 01:09:51",
18
      "violationCount":2,
19
      "ruleName":"Default Page Load Alert Rule",
20
      "agents":[
21
         {
22
            "dateStart":"2014-08-24 01:01:47",
23
            "dateEnd":"2014-08-24 01:09:47",
24
            "active":0,
25
            "metricsAtStart":"Page load: Incomplete",
26
            "metricsAtEnd":"Page Load: Complete",
27
            "agentId":108,
28
            "agentName":"Boston, MA",
29
            "locationName":"Boston, MA",
30
            "permalink":"https://app.thousandeyes.com/alerts/list?__a=11&alertId=35541&agentId=108"
31
         },
32
         {
33
            "dateStart":"2014-08-24 01:01:51",
34
            "dateEnd":"2014-08-24 01:09:51",
35
            "active":0,
36
            "metricsAtStart":"Page load: Incomplete",
37
            "metricsAtEnd":"Page Load: Complete",
38
            "agentId":107,
39
            "agentName":"Vancouver, Canada",
40
            "locationName":"Vancouver, Canada",
41
            "permalink":"https://app.thousandeyes.com/alerts/list?__a=11&alertId=35541&agentId=107"
42
         }
43
      ],
44
      "testTargetsDescription": [
45
         "http://google.com"
46
      ],
47
      "type":"Page Load",
48
      "permalink":"https://app.thousandeyes.com/alerts/list?__a=11&alertId=35541",
49
      "apiLinks":[
50
         {
51
            "rel":"related",
52
            "href":"http://api.thousandeyes.com/tests/5370"
53
         },
54
         {
55
            "rel":"data",
56
            "href":"http://api.thousandeyes.com/web/page-load/5370"
57
         }
58
      ]
59
   }
60
}
Copied!
AGENT_ALERT_NOTIFICATION_TRIGGER
1
{
2
    "eventType":"AGENT_ALERT_NOTIFICATION_TRIGGER",
3
    "eventId":"164522-5744505",
4
    "agentAlert":{
5
        "agentRuleId":43587,
6
        "agentNotificationId":5744505,
7
        "agentId":25694,
8
        "hostname":"TE-AGENT1.local.lan",
9
        "agentName":"Packet1",
10
        "ipAddress":"147.75.199.171",
11
        "active":1,
12
        "ruleExpression":"Clock Offset ≤ 60 seconds",
13
        "ruleName":"Default Agent Clock Offset Notification",
14
        "dateStart":1476987748,
15
        "metricsAtStart":"Clock Offset: 20 seconds",
16
        "metricsAtEnd":""
17
    }
18
}
Copied!
AGENT_ALERT_NOTIFICATION_CLEAR
1
{
2
    "eventType":"AGENT_ALERT_NOTIFICATION_CLEAR",
3
    "eventId":"164534-5003852",
4
    "agentAlert":{
5
        "agentRuleId":43587,
6
        "agentNotificationId":5003852,
7
        "agentId":13558,
8
        "hostname":"cluster-agent2",
9
        "agentName":"cluster-agent2",
10
        "ipAddress":"10.100.20.110",
11
        "active":0,
12
        "ruleExpression":"Clock Offset ≤ 60 seconds",
13
        "ruleName":"Default Agent Clock Offset Notification",
14
        "dateStart":1467175574,
15
        "dateEnd":1476987748,
16
        "metricsAtStart":"Clock Offset: 130 seconds",
17
        "metricsAtEnd":"Clock Offset: 130 seconds"
18
    }
19
}
Copied!
Firewall Exception Requirements
If your web server only allows requests from allow-listed senders, you will need to allow-list requests from ThousandEyes web servers, which come from the IP address 52.52.142.26 and 52.52.36.83.
Related Articles
​How Alerts Work is a detailed guide to ThousandEyes Alerts.
For a guide on creating your own alert rules, see Creating and editing Alert Rules.