Automatic Disabling of Failing Streaming Integrations

Streaming integrations consume resources on the ThousandEyes backend. To maintain efficiency, only functional streaming integrations are supported.

Automatic disabling and failure notifications apply to all OpenTelemetry streaming integrations, including:

• Integrations you create in the ThousandEyes platform (UI or API)

• Integrations created and managed through cross-product integration flows (integrations between products)

Cross-product integrations use different notification wording in the ThousandEyes platform. See Cross-Product Integrations for the notification text.

Criteria for Disabling

If a streaming integration fails to stream data for 12 consecutive hours, you receive a warning notification in the ThousandEyes platform. This notification informs you that the integration will be disabled in 12 hours if the issue is not resolved. You can find your notifications at the bell icon.

If failures continue for 12 consecutive hours, the integration is disabled automatically, and the platform notifies you again.

Successful data export at any point resets the failure counter to zero.

Warning Notification

For integrations you create directly, the warning notification uses this message:

Your OpenTelemetry integration "integrationID" has been failing since date. If the issue is not resolved, this integration will be disabled automatically.

Warning notification for a user-created OpenTelemetry integration

Disabled Notification

For integrations you create directly, the disabled notification uses this message:

This OpenTelemetry integration "integrationID" has been disabled due to continuous failures since date. Please review the issue and re-enable the integration to resume data export.

Disable notification for a user-created OpenTelemetry integration

Cross-Product Integrations

For OpenTelemetry integrations managed through cross-product integration flows, the ThousandEyes platform sends warning and disable notifications with different wording. Notifications refer to a partner OpenTelemetry integration and include guidance to contact the owner or administrator of that integration.

Warning notification:

Your partner OpenTelemetry integration "integrationID" has been failing since date. If the issue is not resolved, this integration will be disabled automatically. Please contact the owner or administrator of the partner integration to review and correct the issue.

Warning notification for a cross-product OpenTelemetry integration

Disabled notification:

This partner OpenTelemetry integration "integrationID" has been disabled due to continuous failures since date. Please contact the owner or administrator of the partner integration to review the issue and re-enable the integration.

Disable notification for a cross-product OpenTelemetry integration

Common Failure Causes

Failures can occur due to:

• Invalid Target URL: The configured URL cannot receive OTLP data.

• Invalid Target Protocol: Incorrect protocol configuration leads to data export attempts using an incorrect protocol.

• Invalid Authentication Headers: Incorrect tokens in headers may lead to HTTP 401 errors, as the data export request fails authentication.

• Invalid Content Type: Missing or incorrect content type headers can cause data export failures.

Re-Enabling a Disabled Integration

Streaming integrations are disabled, not deleted. You can correct configurations and re-enable them.

For cross-product integrations, re-enabling might require coordination with the integration owner or administrator. If you have the required permissions, you can re-enable the integration using the UI or API steps below.

Re-Enabling Via the UI

1. In the ThousandEyes platform, go to Manage > Integrations.

2. Click the disabled integration to open the edit window.

3. Toggle the Integration Status to Enabled.

4. Click Save to apply changes.

Re-Enabling Via the API

Send a PUT HTTP request to enable the integration:

curl -i -XPUT https://api.thousandeyes.com/v7/streams/<stream_id> \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $BEARER_TOKEN" \
-d '{"enabled": "true"}'