topic Dynatrace distributed tracing with Kafka in Open Q&A

Dynatrace distributed tracing with Kafka

sean_cheevers — Tue, 01 Apr 2025 09:20:51 GMT

I have an architecture which consists of producers and consumers with a Kafka broker in the middle. I have enabled the Dynatrace Kafka plugin which has provided some great additional information on monitoring the broker itself.

The current issue is the trace gets lost when a message is published to Kafka. I would like to the ability to see trace from a producer to a consumer in the service flow.

Does Dynatrace support this out of the box or is there some additional configuration required? I would like to hear if anyone has implemented something similar.

Re: Dynatrace distributed tracing with Kafka

skrystosik — Mon, 30 Dec 2019 17:12:46 GMT

It’s not supported at all right now. I don’t know if there is any option to develop such tracing using current development tools we have. You can post a RFE for such case.

Sebastian

Re: Dynatrace distributed tracing with Kafka

sean_cheevers — Mon, 30 Dec 2019 19:40:16 GMT

@Sebastian K. Thanks for the quick response!

Complete lack of knowledge on Dynatrace, would it be possible to instrument the client with code to push the tace data to one agent or through the Dynatrace server api?

Re: Dynatrace distributed tracing with Kafka

skrystosik — Tue, 31 Dec 2019 09:07:22 GMT

Problem with instrumenting kafka would be overhead. This soft is processing massive amount of fast requests. What should be done is detecting somehow communicates that are going to Kafka and returining from them. I don't know if this is possible but this is how Dynatrace is for example tagging messages that are going through IBM Integration bus. There is SDK for tagging MQ requests for unsupported bus, but this will not work for Kafka I suppose (different type of communication). This is something Dynatrace should do (if this is possible). This is why I suggested produce idea.

Sebastian

Re: Dynatrace distributed tracing with Kafka

Julius_Loman — Tue, 07 Jan 2020 12:30:16 GMT

I think the Kafka support was mentioned by @Andreas G. in one of the recent performance clinics (I just don't remember which one).

Last time I checked it was not generally available but I think it was being worked on since I've seen Kafka tracing code in oneagent java libraries.

Re: Dynatrace distributed tracing with Kafka

Julius_Loman — Wed, 08 Jan 2020 14:17:48 GMT

Just found it, seems to be an early access program (probably not public yet).
See video at the 40:07 mark.

Re: Dynatrace distributed tracing with Kafka

skrystosik — Wed, 08 Jan 2020 15:32:32 GMT

Cool! thx for info

Re: Dynatrace distributed tracing with Kafka

janusz_dabrowsk — Mon, 27 Apr 2020 09:19:12 GMT

Hi
Here's description how to configure Kafka consumer to detect flows across Kafka messaging system

https://www.dynatrace.com/support/help/how-to-use-dynatrace/transactions-and-services/configuration/define-messaging-services/

Re: Dynatrace distributed tracing with Kafka

janusz_dabrowsk — Mon, 27 Apr 2020 09:20:12 GMT

Here's description how to configure Kafka consumer to detect flows across Kafka messaging system

https://www.dynatrace.com/support/help/how-to-use-dynatrace/transactions-and-services/configuration/define-messaging-services/

Re: Dynatrace distributed tracing with Kafka

Kishogembali — Wed, 19 Apr 2023 16:31:35 GMT

Hi Janusz,

Thanks for the link - https://www.dynatrace.com/support/help/platform-modules/applications-and-microservices/services/service-detection-and-naming/service-types/define-messaging-services. I was going through this and made the code changes as per the sample given in node.js tab. But, while configuring the settings at Dynatrace, after selecting the handler file, Dynatrace is unable to find the method and then we are not able to move forward. Do you have any idea and help us how to move forward from here ?