topic Why does metrics API try to apply filter to a dimensionless metric? in Dynatrace API

Why does metrics API try to apply filter to a dimensionless metric?

ext-Antti_Elonh — Fri, 25 Mar 2022 09:23:00 GMT

I have a problem trying to configure alerting (custom anomaly detection) where the calculation includes one dimensionless metric and another with the wished dimensions. The same metric expression works great in Data explorer and dashboards but not in alerting because the metrics API tries to apply the filter also to the dimensionless metric.

The metric expression uses Prometheus metrics exposed by cert-manager like this:

(certmanager_certificate_renewal_timestamp_seconds:splitBy("name","namespace","k8s.deployment.name","dt.entity.kubernetes_cluster")/86400)-(certmanager_clock_time_seconds_gauge:splitBy()/86400)

I have checked the Dynatrace server logs and the actual alerting component queries metrics API for each combination with selector as this:

((certmanager_certificate_renewal_timestamp_seconds:splitBy(name,namespace,\"k8s.deployment.name\",\"dt.entity.kubernetes_cluster\")/86400)-(certmanager_clock_time_seconds_gauge:splitBy()/86400)):filter(eq(\"dt.entity.kubernetes_cluster\",KUBERNETES_CLUSTER-4F32D9E28C22C7F1),eq(\"k8s.deployment.name\",cert-manager),eq(name,test-cert),eq(namespace,test-namespace))

If I try that query in Data explorer it fails because it tries to apply the filter also to that dimensionless metric "certmanager_clock_time_seconds_gauge:splitBy()" - it's made dimensionless with that empty splitBy.

I'm just wondering why does Dynatrace even try to apply a filter to a dimensionless metric because it would always fail?

Re: Why does metrics API try to apply filter to a dimensionless metric?

wolfgang_beer — Thu, 24 Mar 2022 11:34:55 GMT

Yes that is a bug we recently encountered in the alerting module. We are working on a fix.

Re: Why does metrics API try to apply filter to a dimensionless metric?

ext-Antti_Elonh — Thu, 24 Mar 2022 13:12:37 GMT

I understand that this can be fixed in the alerting module also but I think there should be a fix in the metrics API as well.

The documentation basically says that a fixed number and a dimensionless metric behave similarly (creating tuples etc.) except that a dimensionless metric is a series. They still don't behave the same regarding that filter transformation. I think they should both be excluded from filter transformation and it might have solved the issue also in this other topic: https://community.dynatrace.com/t5/Dashboarding/I-can-t-create-latency-SLO-using-calculated-metric/m-p/176085

Re: Why does metrics API try to apply filter to a dimensionless metric?

wolfgang_beer — Thu, 24 Mar 2022 15:03:17 GMT

Yes sorry I was not precise enough. We are working on a fix at the moment and the fix is done in the metric query subsystem that includes a fix on API as well as on alerting.

Best greetings,

Wolfgang

Re: Why does metrics API try to apply filter to a dimensionless metric?

ext-Antti_Elonh — Mon, 02 May 2022 08:10:07 GMT

Hi, @wolfgang_beer . Could you tell in which version of a Managed Dynatrace can we expect the fix?

Re: Why does metrics API try to apply filter to a dimensionless metric?

wolfgang_beer — Mon, 02 May 2022 09:01:05 GMT

Hi @ext-Antti_Elonh there are 2 topics still upcoming for DT Managed support which are:

- View the list of problems (precondition is Managed Release 240)

- Push notifications for Managed Environments (expect that in Mai)

Best greetings,

Wolfgang

Re: Why does metrics API try to apply filter to a dimensionless metric?

ext-Antti_Elonh — Tue, 03 May 2022 10:09:13 GMT

Hi,
I was wondering about the fix to metric query problem in anomaly detection.

Re: Why does metrics API try to apply filter to a dimensionless metric?

wolfgang_beer — Tue, 10 May 2022 08:15:05 GMT

According to what I see from dev side, this reported issue was fixed with Dynatrace release 239.

Best greetings,

Wolfgang

Re: Why does metrics API try to apply filter to a dimensionless metric?

ext-Antti_Elonh — Wed, 25 May 2022 13:56:09 GMT

Our managed cluster was updated to v1.240 last night and these custom alerts started to work as expected.

Re: Why does metrics API try to apply filter to a dimensionless metric?

wolfgang_beer — Mon, 30 May 2022 05:51:19 GMT

@ext-Antti_Elonh Excellent, many thanks for sharing this confirmation with us!

Best greetings,

Wolfgang