A name for your custom webhook operation.
We recommend you use a human-readable, meaningful string. If you create multiple operations, it’s a good practice to use a consistent naming convention.
| | **URL** |The target URL of the server associated with your webhook.
Note: HTTP redirects are not accepted. Make sure you provide the final URL in this field.
| | **Preset Configurations** |These optional preset configurations provide a starting place from which you can create your custom webhook. If you use a preset configuration, it populates the fields below with sample content that you can adapt.
Note: These preset configurations can serve as a guide, but you should verify that the resulting webhook’s structure and variables meet your requirements.
You can use any of the following preset configurations for your webhook:
Caution: When you select a preset configuration, any settings you have already made in the fields below are deleted.
| | **Auth Type** |Set the authentication method for your webhook:
For basic and token authentication, see Webhook Authentication.
For OAuth, see Using OAuth Authentication for Your Custom Webhook.
| | **Headers** |The Headers field contains key/value pairs that make up the webhook’s HTTP header to a server and that function as metadata for the webhook.
The key/value pairs you enter here can be static (a set key and its set value) or dynamic (a set key and a variable as its value)
To add additional key/value pairs to the header of your webhook, click + Key-Value Pair and enter the key and value.
To remove a key/value pair, click the - symbol.
| | **URL Query Parameters** |These parameters are appended to a URL to define content or actions, especially when integrating into a third-party system.
For guidance on defining query parameters, see Introduction to Handlebars.
You can type directly into the text box to add URL query parameters to your webhook.
To see an expanded view of the contents of the field, click the arrows button.
| | **Body** |Define the body of your webhook to suit your needs. Note that the body
You can type directly into the text box to customize the body of your webhook.
To see an expanded view of the contents of the field, click the arrows button.
For guidance on the editor for the Body field, see Working in the Webhook Editor.
For convenience, the variables and some guidance on using them are available in Webhook Variables.
| 5. Click **Save**. Your new webhook is listed under the **Operations** tab. To attach alert rules to your new webhook, see the section [Attaching Alert Rules to Your Webhook](#attaching-alert-rules-to-your-webhook). ## Working in the Custom Webhook Operation Editor ### Handlebars Templating Language The ThousandEyes webhook templates use the Handlebars templating language. In fact, in a custom webhook, the **Body** field *is* a Handlebars template. When you write your own webhook, the syntax in the **Body** field must conform to the Handlebars syntax rules. For detailed information on the Handlebars language, see the [Handlebars guide](https://handlebarsjs.com/guide/). ### Hinting and Intellisense When you edit the **Headers**, **URL Query Parameters**, and **Body** fields, the webhook editor UI automatically suggests variables that match the text you have written or that was present in the template. For example, if you type “a” into the editor, it suggests “alerts” as a possible completion. Once a given variable is completed, the editor suggests sub-variables available for it. For example, if the variable completed is “alert.”, the editor suggests all possible sub-variables, such as “id” “rule," and “firstSeen." You can prompt the editor's suggestions manually by typing **Ctrl+space**. ### Custom Webhook Operations Connection Status To see the connection status for your custom webhook operation, navigate to **Manage > Integrations**. Click the **Integrations 2.0** tab. Click on the **Operations** tab. The **Status** column can show the following statuses: | Status | Definition | | --------- | ------------------------------------------------------------------------------------ | | Pending | The integration request has been received and is awaiting further action. | | Connected | The integration has been successfully established and is currently active. | | Failing | The integration exists but its connector status has not been confirmed or validated. | * When an integration is first set up, its status is **Pending**. * Once communication is established, the status updates to **Connected**. * If the integration's state cannot be confirmed, it is marked as **Failing**. ## Testing a Webhook You can test a webhook while you are creating it, or after you have saved it: * The Test button appears at the operation level, not on connectors. * The Test button is also available in the Custom Webhook integration template. * For operations without an associated connector, the button is visible but remains disabled. To test a custom webhook during creation, use the operation configuration panel's **Test** button at the bottom right of the screen. To test an existing custom webhook operation: 1. In the ThousandEyes platform, go to **Manage > Integrations**. From the **Integrations 2.0** tab, select the **Operations** tab. 2. In the list of your operations, find the webhook operation you want to test. The **Status** column shows whether the webhook has been tested. 3. Click the row for the webhook operation you want to test (or click its context menu and select **Edit**). 4. In the **Edit** panel, click **Test**. If your test fails, use the error message as displayed in the **Edit** panel to troubleshoot your configuration. ### How Webhooks Are Tested When you click **Test**, the following elements of your webhook’s configuration are tested: * The URL you have provided. * The authentication you have provided. * The format of the payload. The ThousandEyes platform sends a dummy HTTP alert trigger to test the format of the payload as defined in your webhook. It evaluates the server’s response to determine whether the server can accept the payload. {% hint style="info" %} The **Test** function in this panel is not a foolproof verification of your webhook. The best way to test a webhook is with a real alert. {% endhint %} ### Test Status The **Status** field for **Operations** can have the following values: | **Untested** | This webhook has not been tested through the ThousandEyes platform. | | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Failure** |This webhook has been tested through the ThousandEyes platform, but has failed this test.
If your webhook has a Failure status, it can still be attached to alert rules and might still be dispatched.
The Failure status typically means either that the server did not respond, or that the payload itself is malformed. However, in some cases these failures do not prevent the successful dispatch of a custom webhook.
| | **Success** |This webhook has been tested through the ThousandEyes platform, and has successfully dispatched a test payload.
However, note that in some cases a successful test does not guarantee the successful dispatch of a custom webhook attached to an alert rule.
| ## Attaching Alert Rules to Your Webhook Once you have created your custom webhook operation, you can attach alert rules to it so that ThousandEyes alert notifications can flow into the system you’ve integrated with. To learn more about alert rules, see [Creating and Editing Alert Rules](https://docs.thousandeyes.com/product-documentation/alerts/creating-and-editing-alert-rules). 1. In the ThousandEyes platform, go to **Manage > Integrations** and select the **Operations** tab. 2. In the list of operations, find the custom webhook operation you want to attach alert rules to. 3. Click the vertical dots at the end of the row, and select **Manage Alert Rules** from the dropdown. 4. In the **Manage Alert Rules** panel, select the alert rule or rules you want to associate with this custom webhook operation. If you have a large number of alert rules, use the **Search** box to filter on a relevant string. You can also sort alert rules by name and by alert type. 5. Click **Save**. {% hint style="info" %} You can attach any custom webhook operation to any Cloud and Enterprise Agent alert rule. However, some variables used in a webhook may not be applicable to some alert rules. In these cases, the inapplicable variable is passed as a null value to the system you have integrated with. {% endhint %} ### Failed Alert Notifications If alert notifications fail to be sent via custom webhook operations, we continuously retry sending the notification for up to two hours after the first failure. We retry up to nine times at the following intervals (in minutes) after the first failure: ``` 5 + 10 + 15 + 15 + 15 + 15 + 15 + 15 + 15 ``` These intervals map to the following minutes after the first failure: ``` 5 - 15 - 30 - 45 - 60 - 75 - 90 - 105 - 120 ``` If after this time the notification still isn't sent, the notification is dropped. ## Editing a Custom Webhook Operation To edit a custom webhook operation, see [Integrations](https://docs.thousandeyes.com/product-documentation/integration-guides). ## Deleting a Custom Webhook Operation To delete a custom webhook operation, see [Integrations](https://docs.thousandeyes.com/product-documentation/integration-guides). ## Firewall Exception Requirements If your web server only allows requests from allowlisted senders, you will need to allow requests from ThousandEyes web servers, which come from the following IP addresses, which are region-specific: **US1 Region** ``` 13.56.245.241 52.9.183.148 18.232.232.61 35.168.54.3 107.22.84.44 3.220.243.232 3.218.27.195 3.221.227.188 ``` **US2 Region** ``` 3.141.159.49 3.17.98.26 3.134.227.22 3.18.18.42 3.13.54.169 3.138.52.162 52.27.149.70 52.32.30.54 52.89.210.182 44.227.213.61 35.155.240.202 35.81.172.197 ``` **EU1 Region** ``` 18.157.124.37 3.70.3.30 18.156.17.238 18.158.163.183 35.158.19.241 3.127.8.252 46.51.169.205 54.75.173.76 54.217.22.60 34.243.129.225 54.216.15.243 108.128.60.238 ``` For more information about regions, see [Multi-Region Cloud Support](https://docs.thousandeyes.com/product-documentation/user-management/thousandeyes-multi-region-cloud-support). ## Troubleshooting Your Custom Webhook If you’re encountering issues with your custom webhook, consider the following points: * A common way a webhook might fail despite passing testing is when conditional Handlebars helpers are used. For example, the template might contain a clause like the following: `"timeTriggered": {{#eq type 2 }} {{ alert.firstSeen.epochSecond }} {{/eq}}` If the underlying alert’s state is *triggered*, as it is in the sample we use when we test a webhook, this syntax renders correctly. But if the alert’s state is *cleared*, this syntax fails due to being invalid JSON: the **“timeTriggered”** key in the resulting object will have no value. * Watch for illegal characters in the underlying alert data, particularly for custom headers. We apply URL escaping to query parameters, but not to the webhook's header or payload. * Do not use **#** for custom helpers like **formatDate** or **formatExpression**.