OpenTelemetry and new metric dimension

mkaszub — Fri, 24 Jan 2025 15:08:45 GMT

Hi,

After some time, I decided to add a new dimension (tag) to my metric. However, it appears that Dynatrace is not recognizing it. I can't see the new dimension when attempting to split the metric or in the metric details where all dimensions are listed. For testing purposes, I created an identical metric with a different name, and the additional dimension is visible there. This suggests there might be a caching issue or that the metric definition hasn't been updated.

I haven't found a way to remove the OpenTelemetry metric. What are my options for adding the new dimension to the metric without changing its name?

Regards,
Marcin

Re: OpenTelemetry and new metric dimension

Mizső — Sun, 26 Jan 2025 18:15:56 GMT

Hi @mkaszub

I am not an OTEL expert but, have you checked these domcumentaion topics:

Configure OpenTelemetry metrics ingest — Dynatrace Docs

OpenTelemetry metrics limitations — Dynatrace Docs

I hope it helps.

Best regards,

János

Re: OpenTelemetry and new metric dimension

RohitBisht — Tue, 28 Jan 2025 01:45:30 GMT

Hi @mkaszub ,
Here are few things you can check.
1. Protocol and syntax for dimension: Check if it is following this: https://docs.dynatrace.com/docs/shortlink/metric-ingestion-protocol
2. Test using API and send dummy values onto the metric using that dimension. https://docs.dynatrace.com/docs/shortlink/api-metrics-v2

If point 2 works, maybe there is something wrong with the otel code sending data.

Re: OpenTelemetry and new metric dimension

gopher — Tue, 28 Jan 2025 12:17:48 GMT

Hi mkaszub,

I haven't had any issues adding / removing - example of our processors below.
On changing them (adding too or deleting), The only way I could replicate what you are seeing is by looking within the same time window. after the time window has passed for the last metric captured it won't be shown.

processors: attributes: actions: - key: k8s.cluster.name value: 'my cluster name' action: insert cumulativetodelta: {} filter: metrics: exclude: match_type: expr expressions: - MetricType == "Summary" memory_limiter: check_interval: 5s limit_percentage: 80 spike_limit_percentage: 25 batch/traces: send_batch_size: 5000 send_batch_max_size: 5000 timeout: 60s batch/metrics: send_batch_size: 3000 send_batch_max_size: 3000 timeout: 60s batch/logs: send_batch_size: 1800 send_batch_max_size: 2000 timeout: 60s k8sattributes: auth_type: serviceAccount passthrough: false extract: metadata: - k8s.pod.name - k8s.pod.uid - k8s.deployment.name - k8s.statefulset.name - k8s.daemonset.name - k8s.cronjob.name - k8s.namespace.name - k8s.node.name - k8s.cluster.uid pod_association: - sources: - from: resource_attribute name: k8s.pod.name - from: resource_attribute name: k8s.namespace.name - sources: - from: resource_attribute name: k8s.pod.ip - sources: - from: resource_attribute name: k8s.pod.uid - sources: - from: connection transform: error_mode: ignore trace_statements: - context: resource statements: - set(attributes["dt.kubernetes.workload.kind"], "statefulset") where IsString(attributes["k8s.statefulset.name"]) - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.statefulset.name"]) where IsString(attributes["k8s.statefulset.name"]) - set(attributes["dt.kubernetes.workload.kind"], "deployment") where IsString(attributes["k8s.deployment.name"]) - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.deployment.name"]) where IsString(attributes["k8s.deployment.name"]) - set(attributes["dt.kubernetes.workload.kind"], "daemonset") where IsString(attributes["k8s.daemonset.name"]) - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.daemonset.name"]) where IsString(attributes["k8s.daemonset.name"]) - set(attributes["dt.kubernetes.cluster.id"], attributes["k8s.cluster.uid"]) where IsString(attributes["k8s.cluster.uid"]) log_statements: - context: resource statements: - set(attributes["dt.kubernetes.workload.kind"], "statefulset") where IsString(attributes["k8s.statefulset.name"]) - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.statefulset.name"]) where IsString(attributes["k8s.statefulset.name"]) - set(attributes["dt.kubernetes.workload.kind"], "deployment") where IsString(attributes["k8s.deployment.name"]) - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.deployment.name"]) where IsString(attributes["k8s.deployment.name"]) - set(attributes["dt.kubernetes.workload.kind"], "daemonset") where IsString(attributes["k8s.daemonset.name"]) - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.daemonset.name"]) where IsString(attributes["k8s.daemonset.name"]) - set(attributes["dt.kubernetes.cluster.id"], attributes["k8s.cluster.uid"]) where IsString(attributes["k8s.cluster.uid"])

topic OpenTelemetry and new metric dimension in Open Q&A

OpenTelemetry and new metric dimension

Re: OpenTelemetry and new metric dimension

Re: OpenTelemetry and new metric dimension

Re: OpenTelemetry and new metric dimension