Splunk Cloud Platform and Splunk Enterprise

Splunk Cloud Platform and Splunk Enterprise are data analytics platforms that collect, index, and analyze machine data from any source to deliver operational intelligence. They support ingesting OpenTelemetry data as events or metrics for real-time monitoring, search, and visualization. This guide explains how to integrate ThousandEyes data with Splunk Cloud Platform or Splunk Enterprise using OpenTelemetry.

Prepare Your Splunk Instance

To configure ThousandEyes for Splunk, ensure that the HTTP Event Collector (HEC) is enabled in your Splunk instance. Follow these resources for more information:

• Configure HTTP Event Collector on Splunk Cloud Platform

• Configure HTTP Event Collector on Splunk Enterprise

After enabling the HTTP Event Collector (HEC), follow these steps to prepare your Splunk instance:

1. (Optional) Create an index to specify whether the data will be received as events or metrics. For detailed instructions on creating an index, see Splunk: Set up multiple indexes.

   
2. Create an HTTP Event Collector (HEC) token in your Splunk instance.

3. (Optional) Associate the token with the created index for data routing.

4. Identify the target endpoint. Use the appropriate target endpoint based on your Splunk deployment:

   • Splunk Cloud Platform:

     • events: https://http-inputs-<host>.splunkcloud.com:443/services/collector/event

     • metrics: https://http-inputs-<host>.splunkcloud.com:443/services/collector

   • Splunk Enterprise:

     • events: https://<host>:8088/services/collector/event

     • metrics: https://<host>:8088/services/collector

Note: ThousandEyes for OpenTelemetry currently does not support Splunk trial accounts due to an issue with TLS self-signed certificates.

Create an Integration

Create an Integration Using the ThousandEyes UI

Step 1: Create a Connector

To create a new integration, do the following:

1. In the ThousandEyes platform, go to Manage > Integrations > Integrations 2.0.

2. Click + New Connector to select the type of a connector to configure.

3. Click Splunk Cloud Platform HEC or Splunk Enterprise HEC depending on your target.

   
4. Fill out the following mandatory fields:

   • Name: A name for your connector.

   • Target: The target URL of the integration, which may include the port, such as https://http-inputs-<host>.splunkcloud.com:443/services/collector/event for Splunk Cloud Platform or https://<splunk_hostname>/services/collector/event for Splunk Enterprise.

   • Note: When you create a stream, the target URL must satisfy the Stream endpoint URL requirements.

   • Token: Enter the Splunk HEC token.

     
5. Click Save & Assign Operation to save the connector. This connector is now visible in the list view (Connectors tab).

Step 2: Create an Operation

After you create a connector, set up an operation to stream data to the target and assign it to the connector.

1. Click + New Operation to open the menu for selecting the operation type.

2. Choose Splunk Enterprise, Splunk Cloud Platform to proceed to the configuration form.

3. Fill out the configuration form:

   • Mandatory fields:

     • Operation Name: A name for your operation.

   • Optional fields:

     • Index: Specify the Index on your Splunk instance where the data will be stored.

     • Source: Define the source of the data.

     • Source Type: Specify the source type of the data.

     • OpenTelemetry Signal: Select how the telemetry data will be sent to the target.

     • Integration Status: Toggle the integration status to either enable or disable the integration upon creation.

4. Under Test Data Configuration, configure what data is streamed by selecting tests and tags associated with your current account group:

   • Network & App Synthetics Tests: Select Network & App Synthetics test data available in your account group to stream to the target.

   • Endpoint Experience Tests: Select Endpoint Experience test data from your account group to stream to the target.

   • Tags: Select tags from your current account group to associate with the integration. Data from Network & App Synthetics tests and Endpoint Experience tests tagged with the same tags you select in this dropdown will be streamed to the configured target.

   For more information on creating a test, see General Setup Instructions.

Step 3: (Optional) Test Communication with the Target

You can test the connection to your target to ensure it is configured correctly.

1. Run the test:

   • Click Test at the bottom of the configuration form.

   • A test message is sent to the target to verify the connection.

2. Review the test results:

   • If the test is successful:

     • A success message is displayed.

       
   • If the test fails:

     • An error message is displayed with details about the failure.

       

Step 4: Save the Operation

After you complete the form and testing, click Save to complete the creation of the integration. The operation is now visible in the list view (Operations tab).

Manage Integrations in the UI

For more information on managing Splunk Observability Cloud or Enterprise OpenTelemetry integrations, including listing, editing, and deleting integrations, see Manage Integrations Using the UI - Integrations 2.0.

Create an Integration Using the ThousandEyes API

For a programmatic integration, use the following API command:

HTTP Integration:

curl -i -XPOST https://api.thousandeyes.com/v7/stream -H "Content-Type: application/json" -H "Authorization: Bearer $BEARER_TOKEN" -d '{
 "type": "splunk-hec",
 "testMatch": [
     {
       "id": "987654",
       "domain": "cea"
     }
 ],
 "endpointType": "http",
 "streamEndpointUrl": "https://http-inputs-{HOST}.splunkcloud.com:443/services/collector/event",
 "exporterConfig" : {
   "splunkHec": {
     "token": "{TOKEN}",
   }
 }
}'

When you create a stream from Splunk Enterprise or Splunk Cloud, ensure the following:

• The type is "splunk-hec".

• The endpointType is "http" .

• exporterConfig.splunkHec.token is set to the value of Splunk HEC Token.

• In case you had created an index and it is associated with the HEC token, when creating a stream, specify the name of the index in the exporterConfig.splunkHec.index field.

• The streamEndpointUrl satisfies the Stream endpoint URL requirements.

Receiving data in Splunk

Now, you can start receiving ThousandEyes data in Splunk products

Receiving ThousandEyes data as events

Search using the Query source=ThousandEyesOTel or the index index="thousandeyes_otel_events_index".

Receiving ThousandEyes data as metrics

Search the metrics using the Query | mcatalog values(metric_name) WHERE index=* or the values of a metrics | mstats avg(_value) WHERE index=* AND metric_name=network.latency span=30s. For more information, see Splunk: Search and monitor metrics.