topic Re: Prometheus histogram metrics in Automations

Prometheus histogram metrics

JohnDoe — Thu, 13 Jul 2023 17:18:44 GMT

Hey all,

as per the current documentation, the Prometheus integration does only support counter and gauge metrics but not histograms:

https://www.dynatrace.com/support/help/how-to-use-dynatrace/infrastructure-monitoring/container-platform-monitoring/kubernetes-monitoring/monitor-prometheus-metrics#limitations

Are there any plans or roadmaps to support Prometheus' histogram metrics? Or any proposed workaround to still get this data into Dynatrace? We have some components that expose histogram metrics by default and we consider these as very relevant for us to be monitored.

Thanks in advance!

Re: Prometheus histogram metrics

Michal_Gebacki — Tue, 04 Jan 2022 12:12:47 GMT

Hello guys!

Does anyone have some tips regarding subject detailed above? Would be awesome, thanks in advance! 😄

Re: Prometheus histogram metrics

Mike_L — Tue, 04 Jan 2022 12:18:36 GMT

@Lukasz_Be would know what the plan is moving forward. Currently we indeed only support gauge and counter.

A workaround would be to "massage" the data in a script or such and then send it to Dynatrace using the metric ingest API.

Re: Prometheus histogram metrics

Lukasz_Be — Fri, 21 Jan 2022 09:29:27 GMT

Hi @JohnDoe,
Yes, we do have a plan to introduce the support for Prometheus histograms.

Re: Prometheus histogram metrics

JohnDoe — Fri, 21 Jan 2022 10:33:45 GMT

Hey @Lukasz_Be , glad to hear this and looking forward to it. Is there any (rough) roadmap for this feature you can share?

Re: Prometheus histogram metrics

Lukasz_Be — Wed, 26 Jan 2022 11:10:21 GMT

Hi, I am afraid I can't share the details but it is on our mid-term roadmap.

Re: Prometheus histogram metrics

henk_stobbe — Mon, 07 Feb 2022 18:00:52 GMT

Hello,

Just looked at the extension log:

Could not get value of Prometheus metric because of unsupported type. Supported Prometheus types are: counter, gauge, untyped.

So untyped, anybody know or has this something to do with above histogram (please (-;)

KR Henk

(And counter should be count I hope)

Re: Prometheus histogram metrics

Arkadiusz_Banas — Wed, 09 Feb 2022 09:13:44 GMT

Hi,

The untyped has nothing to do with histograms, sorry. That log mentions Prometheus types meaning the types that can be found in Prometheus exporters. Here's a link to Prometheus docs: https://prometheus.io/docs/instrumenting/writing_exporters/. There is a section dedicated to types which describes what untyped is and what is it for. Also, counter metric in Prometheus is called counter while in Dynatrace we have count (see https://prometheus.io/docs/concepts/metric_types/ and https://www.dynatrace.com/support/help/how-to-use-dynatrace/metrics/metric-ingestion/metric-ingestion-protocol#count-metric).

If you have any further questions, don't hesitate to ask me.

Best wishes,

Arek.

Re: Prometheus histogram metrics

JohnDoe — Fri, 11 Feb 2022 11:51:36 GMT

ok, thanks for the feedback 🙂

Re: Prometheus histogram metrics

DanielS — Wed, 23 Mar 2022 18:54:55 GMT

Thanks @Lukasz_Be any update about this that you can share with us? I have several clients asking for histograms, it would be great if we can have a non compromise release month. Thanks

Re: Prometheus histogram metrics

Roland — Fri, 05 Aug 2022 20:40:14 GMT

Was wondering is it midterm calendar or fiscal year? In any case would you mind providing current status please?

Re: Prometheus histogram metrics

JohnDoe — Tue, 27 Sep 2022 09:30:32 GMT

@Lukasz_Be any news? A short update would be highly appreciated.

Re: Prometheus histogram metrics

dglugla — Tue, 27 Sep 2022 10:14:00 GMT

@JohnDoe , @Roland , @DanielS & @henk_stobbe : in a meantime "mid-term" became a "short-term" plan, so the current ETA for the histogram metrics support is the beginning of CQ1'23. There was a huge step to take before we can get into the development here, so sorry for the delay.

Re: Prometheus histogram metrics

JohnDoe — Tue, 27 Sep 2022 10:25:59 GMT

many thanks for the quick response, looking forward to CQ1'23 🙂

Re: Prometheus histogram metrics

dglugla — Mon, 17 Oct 2022 14:56:28 GMT

turns out that the histogram metrics support will be limited to only the "Summary" part. Our metric ingest and storage is not yet ready to handle the histogram buckets.

Re: Prometheus histogram metrics

Roland — Mon, 17 Oct 2022 15:20:05 GMT

When will it be ready to support the histogram buckets?

Re: Prometheus histogram metrics

dglugla — Mon, 17 Oct 2022 15:48:58 GMT

I'm afraid I can't answer that. Maybe @lkoniecki or @Herwig_Moser can provide more details?

Re: Prometheus histogram metrics

DanielS — Thu, 23 Mar 2023 13:29:56 GMT

Well, great news. Now it is working with lastest AG release!!!!

https://www.dynatrace.com/support/help/shortlink/release-notes-activegate-sprint-261#prometheus-histogram-metrics

Re: Prometheus histogram metrics

justbalazs — Fri, 19 May 2023 08:48:04 GMT

Hi!

We use Active Gate 1.263, version (https://www.dynatrace.com/support/help/whats-new/release-notes/activegate/sprint-261#prometheus-histogram-metrics) but Prometheus histogram metrics does not work for us.

We try to ingest these Prometheus metrics (via a custome made DT plugin) into Dynatrace.

- key: com.dynatrace.extension.prometheus-matrix.global.mt_activity_compensate_duration_sum

value: metric:mt_activity_compensate_duration_sum

type: count

featureSet: global

- key: com.dynatrace.extension.prometheus-matrix.global.mt_activity_compensate_duration_count

value: metric:mt_activity_compensate_duration_count

type: count

featureSet: global

We can see the age and count metrics in the Data Explorer, but we can not see these histogram metrics in it.

Thanks for your help!

Re: Prometheus histogram metrics

danielywoo — Thu, 29 Jun 2023 04:08:49 GMT

I have exactly the same issue, there is no such "summary" or "histogram" in DynaTrace. But I think we probably can use "advanced query" to work around, in your case, it can be:

(mt_activity_compensate_duration_sum)
/
(mt_activity_compensate_duration_count)