Configuring ThousandEyes for Splunk Cloud or Enterprise using the API

This section describes how to create a data stream using ThousandEyes for OpenTelemetry API to Splunk Cloud Platform or Splunk Enterprise. This enables you to start receiving ThousandEyes data in Splunk products.

Splunk offers three observability products:

Splunk Observability Cloud
- Receives data in OpenTelemetry format.
- Configure it using ThousandEyes for OpenTelemetry.
Splunk Cloud Platform and Splunk Enterprise
- These products support receiving telemetry data in their proprietary HEC Splunk format only.

Note: Integration with Splunk products is available only via the ThousandEyes for OpenTelemetry API.

Prerequisites

To use the ThousandEyes API, make sure you meet the following requirements:

Your user role must have the following permissions:
- API access permission. The three built-in roles (Organization Admin, Account Admin, and Regular User) include this permission by default.
- Edit streaming integrations, to view and manage integrations.
- View streaming integrations, to view details about integrations.
You must have a user API token generated by the ThousandEyes platform to authenticate your requests. For more details, see the ThousandEyes Developer Reference.

Step 1: Set Up a Network Test

This step is optional. If you have already set up a test to configure your data stream, call the test list endpoint to get a list of tests. Copy the test's testId for use in configuring the data stream. Once you have the testId, proceed to Step 2.

Follow these steps to set up a network test under CEA (Cloud and Enterprise Agents):

Check which agents are available:

curl -i -XGET https://api.thousandeyes.com/v7/agents -H "Authorization: Bearer $BEARER_TOKEN"

The response should contain a list of available agents:

{ 
  [
    {
      "agentId": 12345,
      "agentName": "Tests",
      "agentType": "Cloud",
      "countryId": "US",
      "targetOnly": 0,
      "ipAddresses": [
          "128.0.0.1"
      ],
      "location": "San Francisco Area",
      "createdDate": "2023-01-25 22:01:45"
    },
    ...
  ]
}

From the list of available agents, select one Cloud Agent and use its agentId to create a test:

curl -i -XPOST https://api.thousandeyes.com/v6/tests/agent-to-server/new.json -H "Authorization: Bearer $BEARER_TOKEN" -d '{
        "interval": 60,
        "agents": [
            {"agentId": 12345}
        ],
        "testName": "Opentelemetry Test",
        "server": "www.thousandeyes.com",
        "port": 80,
        "alertsEnabled": 0
    }'

The response contains a testId field. Set this value aside for use in assigning a tag to this test in subsequent steps.

{
"test": [
    {
    "enabled": 1,
    "testId": 987654,
    "testName": "Opentelemetry Test",
    "type": "agent-to-server"
    ...
    }
  ]
}

Step 2: Get the Splunk HEC Token and Target

Follow the Splunk documentation to set up and use the HTTP HEC Event Collector in Splunk Web. This includes these steps:

Optionally. Create an index where you can specify if you will receive the data as events or metrics.
Create an HEC token. Optionally, associate it with the created index.
Identify the target endpoint.
- For Splunk Cloud Platform, use:
  - events: https://http-inputs-<host>.splunkcloud.com:443/services/collector/event
  - metrics: https://http-inputs-<host>.splunkcloud.com:443/services/collector
- For Splunk Enterprise, use:
  - events: https://<host>:8088/services/collector/event
  - metrics: https://<host>:8088/services/collector

Note: ThousandEyes for OpenTelemetry does not currently support Splunk trial accounts, due to an issue with TLS self-signed certificates.

Step 3: Create a Stream

To create a stream, send a request to the /v7/stream endpoint with the target endpoint details and the CEA (Cloud and Enterprise Agents) test's ID and domain, matching the details from the previous step:

curl -i -XPOST https://api.thousandeyes.com/v7/stream -H "Content-Type: application/json" -H "Authorization: Bearer $BEARER_TOKEN" -d '{
  "type": "splunk-hec",
  "testMatch": [
      {
        "id": "987654",
        "domain": "cea"
      }
  ],
  "endpointType": "http",
  "streamEndpointUrl": "https://http-inputs-<host>.splunkcloud.com:443/services/collector/event",
  "exporterConfig" : {
    "splunkHec": {
      "token": "<TOKEN>",
      "index": "<INDEX>"
    }
  }
}'

When you create a stream, the streamEndpointUrl must satisfy the Stream endpoint URL requirements.

The response contains the details of the stream you've just created:

{
  "id": "79c7f72e-2e10-427d-b3e9-43d0d422ecfe",
  "enabled": true,
  "type": "splunk-hec",
  "signal": "metric",
  "dataModelVersion": "v2",
  "endpointType": "http",
  "streamEndpointUrl": "https://http-inputs-<host>.splunkcloud.com:443/services/collector/event",
  "testMatch": [
      {
        "id": "987654",
        "domain": "cea"
      }
  ],
  "exporterConfig" : {
    "splunkHec": {
      "token": "******",
      "index": "<INDEX>",
      "source" : "ThousandEyesOTel",
      "sourceType": "ThousandEyesOTel"
    }
  },
  "auditOperation": {
      "createdDate": 1721648143513,
      "createdBy": 84157
  },
  "_links": {
      "self": {
          "href": "https://api.thousandeyes.com/v7/stream/79c7f72e-2e10-427d-b3e9-43d0d422ecfe"
      }
  }
}

When you create a stream from Splunk Enterprise or Splunk Cloud, ensure that

The type is "splunk-hec".
The endpointType is "http" .
exporterConfig.splunkHec.token is set to the value of Splunk HEC Token.
In case you had created an index and it is associated with the HEC token, when creating a stream, specify the name of the index in the exporterConfig.splunkHec.index field.

Step 4: Receive the data in Splunk

Now, start receiving ThousandEyes data in Splunk products

Receiving ThousandEyes data as events:

You can search using the Query source=ThousandEyesOTel or the index index="thousandeyes_otel_events_index"

Receiving ThousandEyes data as metrics

You can search the metrics using the Query | mcatalog values(metric_name) WHERE index=* or the values of a metrics | mstats avg(_value) WHERE index=* AND metric_name=network.latency span=30s, More info.

PreviousConfiguring ThousandEyes for OpenTelemetry Using the UI NextThousandEyes for OpenTelemetry Data Model

Last updated 21 days ago