> For the complete documentation index, see [llms.txt](https://docs.thousandeyes.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.thousandeyes.com/product-documentation/integration-guides/custom-webhook-examples/splunk-alert-notifs.md).

# Splunk Alert Notification

This section explains how to receive ThousandEyes alerts in Splunk using a custom webhook connector and operation.

## Set Up Splunk

1. Log in to **Splunk**.
2. Create an **HTTP Event Collector (HEC) token** in your Splunk instance:
   * [Configure HTTP Event Collector on Splunk Cloud Platform](https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_Splunk_Cloud_Platform)
   * [Configure HTTP Event Collector on Splunk Enterprise](https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_Splunk_Enterprise)
3. Identify the target endpoint based on your Splunk deployment:
   * **Splunk Cloud Platform:**

     ```
     https://http-inputs-<host>.splunkcloud.com:443/services/collector/event
     ```
   * **Splunk Enterprise:**

     ```
     https://<host>:8088/services/collector/event
     ```

## Set Up ThousandEyes

### Create a Connector and Operation

1. In the ThousandEyes platform, go to **Manage > Integrations**.
2. Go to **Integrations 2.0 > Integration Templates**.

   ![Custom Webhook template in the Integration Templates tab](/files/dNQc7jQmj63nV3vv3NKx)
3. Select **Custom Webhook**.
4. Configure the connector:
   * **Name**: Enter a descriptive name for the Splunk connector.
   * **Target**: Use the target endpoint identified in [Set Up Splunk](#set-up-splunk).
   * **Auth Type**: Select the **Custom** authentication type.
   * **Custom Headers**: Add an `Authorization` header with the value `Splunk <HEC Token>`.
5. Click **Save & Assign Operation**.
6. Configure the custom webhook operation:
   * **Operation Name**: Enter a descriptive name for the operation.
   * **Preset Configurations**: Select **Splunk**.
   * **Custom Headers**: Add a `Content-Type` header with the value `application/json`.
   * **Body**: Review the Splunk preset payload.
7. Click **Test** to verify the operation:
   * If the test succeeds, ThousandEyes displays a confirmation message.
   * If the test fails, verify that the HEC target, HEC token, headers, and payload are correct.
8. Click **Save Integration**.

![Splunk custom webhook connector and operation configuration](/files/6Nm9vy6CEhUvDD48eHda)

### Attach Alert Rules to the Operation

1. In the ThousandEyes platform, go to **Manage > Integrations**.
2. Go to **Integrations 2.0 > Operations**.
3. Find the Splunk custom webhook operation.
4. Click the actions menu (⋮) at the end of the row, then select **Manage Alert Rules**.
5. In the **Manage Alert Rules** panel, select the alert rules you want to send to Splunk.
6. Click **Save**.

![Manage Alert Rules panel with the Splunk custom webhook operation selected](/files/2wGfPzPXUDpGbQwtIYy3)

## Receive Alerts in Splunk

1. Log in to **Splunk**.
2. When an alert is triggered, search for the event using the following query: `index="*" eventType="THOUSANDEYES_ALERT_NOTIFICATION"`

**Sample Output:**

```json
{
  "eventId": "0-0",
  "eventType": "THOUSANDEYES_ALERT_NOTIFICATION",
  "id": "0",
  "type": "2",
  "accountId": "0",
  "orgId": "0",
  "testId": "0",
  "thousandeyes_test_id": "0",
  "test_description": "Sample Description",
  "test_type": "HTTP",
  "itsiDrilldownURI": "https://app.thousandeyes.com/network-app-synthetics/views/?testId=0",
  "severity_id": "1",
  "vendor_severity": "INFO",
  "app": "THOUSANDEYES",
  "src": "Sample Target",
  "signature": "Sample Rule",
  "alert_type": "Http",
  "alert": {
    "id": "0",
    "type": "Http",
    "severity": "INFO",
    "test": {
      "name": "Sample Test"
    },
    "targets": [
      "Sample Target"
    ],
    "rule": {
      "id": "0",
      "name": "Sample Rule",
      "expression": "Response Time ≥ 111 ms",
      "notes": "Sample Notes"
    },
    "triggered": 1738322223247,
    "cleared": 1738322223247,
    "details": [
      {
        "metricsAtStart": "Response Time: 888 ms",
        "source": {
          "id": "0",
          "name": "Sample Agent 1"
        }
      },
      {
        "metricsAtStart": "null",
        "metricsAtEnd": "Response Time: 999 ms",
        "source": {
          "id": "0",
          "name": "Sample Agent 2"
        }
      }
    ]
  }
}
```

![Splunk search results showing a ThousandEyes alert event](/files/OkqpS4ekWkcLfEVfEInk)

### Integration with Splunk IT Service Intelligence

Some fields, such as `itsiDrilldownURI` and `app`, are required for [Splunk IT Service Intelligence (ITSI) application](https://splunkbase.splunk.com/app/1841/). Splunk ITSI will receive ThousandEyes alerts, analyze them, and aggregate them with other events.

For more information on configuring webhooks, see [Custom Webhooks](https://docs.thousandeyes.com/product-documentation/integration-guides/custom-webhooks).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.thousandeyes.com/product-documentation/integration-guides/custom-webhook-examples/splunk-alert-notifs.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.