Prometheus histogram metrics

[JohnDoe]

Hey all,

as per the current documentation, the Prometheus integration does only support counter and gauge metrics but not histograms:

https://www.dynatrace.com/support/help/how-to-use-dynatrace/infrastructure-monitoring/container-plat...

Are there any plans or roadmaps to support Prometheus' histogram metrics? Or any proposed workaround to still get this data into Dynatrace? We have some components that expose histogram metrics by default and we consider these as very relevant for us to be monitored.

Thanks in advance!

[Michal_Gebacki]

Hello guys!

Does anyone have some tips regarding subject detailed above? Would be awesome, thanks in advance! 😄

[Mike_L]

@Lukasz_Be would know what the plan is moving forward. Currently we indeed only support gauge and counter.

A workaround would be to "massage" the data in a script or such and then send it to Dynatrace using the metric ingest API.

[Lukasz_Be]

Hi @JohnDoe,
Yes, we do have a plan to introduce the support for Prometheus histograms.

[JohnDoe]

Hey @Lukasz_Be , glad to hear this and looking forward to it. Is there any (rough) roadmap for this feature you can share?

[Lukasz_Be]

Hi, I am afraid I can't share the details but it is on our mid-term roadmap.

[JohnDoe]

ok, thanks for the feedback 🙂

[DanielS]

Thanks @Lukasz_Be any update about this that you can share with us? I have several clients asking for histograms, it would be great if we can have a non compromise release month. Thanks

[Roland]

Was wondering is it midterm calendar or fiscal year? In any case would you mind providing current status please?

[JohnDoe]

@Lukasz_Be any news? A short update would be highly appreciated.

[dglugla]

@JohnDoe , @Roland , @DanielS & @henk_stobbe : in a meantime "mid-term" became a "short-term" plan, so the current ETA for the histogram metrics support is the beginning of CQ1'23. There was a huge step to take before we can get into the development here, so sorry for the delay.

[JohnDoe]

many thanks for the quick response, looking forward to CQ1'23 🙂

[dglugla]

turns out that the histogram metrics support will be limited to only the "Summary" part. Our metric ingest and storage is not yet ready to handle the histogram buckets.

[Roland]

When will it be ready to support the histogram buckets?

[dglugla]

I'm afraid I can't answer that. Maybe @lkoniecki or @Herwig_Moser can provide more details?

[henk_stobbe]

Hello,

Just looked at the extension log:

Could not get value of Prometheus metric because of unsupported type. Supported Prometheus types are: counter, gauge, untyped.

So untyped, anybody know or has this something to do with above histogram (please (-;)

KR Henk

(And counter should be count I hope)

[Arkadiusz_Banas · Accepted solution]

Hi,

The untyped has nothing to do with histograms, sorry. That log mentions Prometheus types meaning the types that can be found in Prometheus exporters. Here's a link to Prometheus docs: https://prometheus.io/docs/instrumenting/writing_exporters/. There is a section dedicated to types which describes what untyped is and what is it for. Also, counter metric in Prometheus is called counter while in Dynatrace we have count (see https://prometheus.io/docs/concepts/metric_types/ and https://www.dynatrace.com/support/help/how-to-use-dynatrace/metrics/metric-ingestion/metric-ingestio...).

If you have any further questions, don't hesitate to ask me.

Best wishes,

Arek.

[DanielS · Accepted solution]

Well, great news. Now it is working with lastest AG release!!!!

https://www.dynatrace.com/support/help/shortlink/release-notes-activegate-sprint-261#prometheus-hist...

[balazs_bakai]

Hi!

We use Active Gate 1.263, version (https://www.dynatrace.com/support/help/whats-new/release-notes/activegate/sprint-261#prometheus-hist...) but Prometheus histogram metrics does not work for us.

We try to ingest these Prometheus metrics (via a custome made DT plugin) into Dynatrace.

          - key: com.dynatrace.extension.prometheus-matrix.global.mt_activity_compensate_duration_sum

            value: metric:mt_activity_compensate_duration_sum

            type: count

            featureSet: global

          - key: com.dynatrace.extension.prometheus-matrix.global.mt_activity_compensate_duration_count

            value: metric:mt_activity_compensate_duration_count

            type: count

            featureSet: global

We can see the age and count metrics in the Data Explorer, but we can not see these histogram metrics in it.

Thanks for your help!

[danielywoo]

I have exactly the same issue, there is no such "summary" or "histogram" in DynaTrace. But I think we probably can use "advanced query" to work around, in your case, it can be:

(mt_activity_compensate_duration_sum)
/
(mt_activity_compensate_duration_count)

[mreider]

I'd like to refer folks to this feedback thread regarding Prometheus and Kubernetes if you have further input.

https://community.dynatrace.com/t5/Feedback-channel/Feedback-Channel-for-Prometheus-Monitoring-in-Ku...