on 05 Feb 2025 10:12 AM
Dynatrace currently does not support cumulative OpenTelemetry metrics aggregation. This article describes what kinds of options you have to configure delta temporality in your services or in an OTel collector.
You will notice that your temporality is not set up correctly through specific log messages and either outright rejected requests or requests that return with “partial success”. The log message will come up wherever you send your OTLP metrics from, be it an OTel SDK or an OTel Collector.
If your request only contains OTLP metrics in cumulative temporality, then you will get an HTTP 400 response with a log message like the following:
All metric data points were rejected. The following issues were encountered while ingesting OTLP metrics:
Errors:
Unsupported metric: 'mymetric' - Reason: UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM
If your request also contained supported OTLP data, you'll get an HTTP 200 response and a log message like the following:
The following issues were encountered while ingesting OTLP metrics:
Errors:
Unsupported metric: 'mymetric' - Reason: UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM
You'll also see the dsfm:active_gate.metrics.ingest.otlp.datapoints.rejected.count self-monitoring metric increase. You can split it by “reason” to see if the UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM error type is there. Unfortunately, it’s not possible to show the exact metric that caused the error here, only the count of errors for the corresponding error type.
To resolve this issue, you need to set the metrics temporality to delta instead of cumulative.
You can either set the correct temporality directly in your services (which is the preferred way) or in an OTel Collector that processes your OTLP metrics and forwards them to Dynatrace.
If the OTel SDK for your programming language supports it, you can set the temporality by setting the OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta environment variable. Find out if your OTel SDK supports this by checking out the support matrix here.
If your OTel SDK does not support the environment variable, then you can set it directly in code, where you set up the metrics SDK. An exampe of how to do that in Python can be seen in the screenshot below and also be found on the documentation page.
If you don’t have the option to use either of the solutions above, you can alternatively also use an OpenTelemetry Collector like the Dynatrace OTel Collector Distribution to convert cumulative metrics to delta temporality. You can find instructions on how to correctly set up your configuration on the documentation page.
If you have any further questions or encounter any issues not listed above, please feel free to contact our support team.