https://community.dynatrace.com/t5/Troubleshooting/How-to-set-up-OpenTelemetry-metrics-with-delta-temporality/ta-p/269292 <H1>Problem</H1> <P>Dynatrace currently <A href="https://docs.dynatrace.com/docs/shortlink/opentelemetry-metrics-limitations#aggregation-temporality" target="_blank" rel="noopener">does not support cumulative OpenTelemetry metrics aggregation</A>. This article describes what kinds of options you have to configure delta temporality in your services or in an OTel collector.<BR /><BR /></P> <H1>Troubleshooting Steps</H1> <P>You will notice that your temporality is not set up correctly through specific log messages and either outright rejected requests or requests that return with “partial success”. The log message will come up wherever you send your OTLP metrics from, be it an OTel SDK or an OTel Collector.</P> <P>If your request only contains OTLP metrics in cumulative temporality, then you will get an HTTP 400 response with a log message like the following:</P> <LI-CODE lang="markup">All metric data points were rejected. The following issues were encountered while ingesting OTLP metrics: Errors: Unsupported metric: 'mymetric' - Reason: UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM</LI-CODE> <P>&nbsp;</P> <P>If your request also contained supported OTLP data, you'll get an HTTP 200 response and a log message like the following:</P> <LI-CODE lang="markup">The following issues were encountered while ingesting OTLP metrics: Errors: Unsupported metric: 'mymetric' - Reason: UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM</LI-CODE> <P>&nbsp;</P> <P>You'll also see the <FONT face="courier new,courier">dsfm:active_gate.metrics.ingest.otlp.datapoints.rejected.count</FONT> self-monitoring metric increase. You can split it by “reason” to see if the <FONT face="courier new,courier">UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM</FONT> error type is there. Unfortunately, it’s not possible to show the exact metric that caused the error here, only the count of errors for the corresponding error type.<BR /><BR /></P> <H1>Resolution</H1> <P>To resolve this issue, you need to set the metrics temporality to delta instead of cumulative.</P> <P>You can either set the correct temporality directly in your services (which is the preferred way) or in an OTel Collector that processes your OTLP metrics and forwards them to Dynatrace.</P> <P>If the OTel SDK for your programming language supports it, you can set the temporality by setting the <FONT face="courier new,courier">OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta</FONT> environment variable. Find out if your OTel SDK supports this by checking out the support matrix <A href="https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#environment-variables" target="_blank" rel="noopener">here</A>.</P> <P>If your OTel SDK does not support the environment variable, then you can set it directly in code, where you set up the metrics SDK. An exampe of how to do that in Python can be seen in the screenshot below and also be found on the <A href="https://docs.dynatrace.com/docs/ingest-from/opentelemetry/walkthroughs/python/python-manual" target="_blank" rel="noopener">documentation page</A>.</P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="mowies_0-1738748258296.png" style="width: 999px;"><img src="https://community.dynatrace.com/t5/image/serverpage/image-id/26208i78BBB1EFD22551B1/image-size/large?v=v2&amp;px=999" role="button" title="mowies_0-1738748258296.png" alt="mowies_0-1738748258296.png" /></span></P> <P>If you don’t have the option to use either of the solutions above, you can alternatively also use an OpenTelemetry Collector like the <A href="https://docs.dynatrace.com/docs/ingest-from/opentelemetry/collector" target="_blank" rel="noopener">Dynatrace OTel Collector Distribution</A> to convert cumulative metrics to delta temporality. You can find instructions on how to correctly set up your configuration on the <A href="https://docs.dynatrace.com/docs/ingest-from/opentelemetry/collector/configuration#delta-metrics" target="_blank" rel="noopener">documentation page</A>.<BR /><BR /></P> <H1>Further Support</H1> <P>If you have any further questions or encounter any issues not listed above, please feel free to contact our&nbsp;<A href="https://support.dynatrace.com/" target="_blank" rel="noopener">support team</A>.</P> Wed, 05 Feb 2025 10:12:44 GMT mowies 2025-02-05T10:12:44Z https://community.dynatrace.com/t5/Troubleshooting/How-to-set-up-OpenTelemetry-metrics-with-delta-temporality/ta-p/269292 <P>Find out how to set up OpenTelemetry metrics with delta temporality</P> Wed, 05 Feb 2025 10:12:44 GMT https://community.dynatrace.com/t5/Troubleshooting/How-to-set-up-OpenTelemetry-metrics-with-delta-temporality/ta-p/269292 mowies 2025-02-05T10:12:44Z