My company has an application that leverages about 60 % java kafka and 40% Go Kafka messaging. We have the OneAgent in our env and on all the microservices but DT Go Kafka is not supported today. Anyone here try to instrument this another way? Using OTEL or OA SDK and have any recommendations for us to be able to trace request from the front end all the way back?
If Dynatrace's official instrumentation for Go Kafka isn't supported in your environment, you can consider alternative approaches to instrumenting your Go Kafka microservices for distributed tracing. While Dynatrace provides OneAgent and SDKs for certain languages like Java, Go might not have native support yet. Here's a general approach you can consider:
1. **OpenTelemetry (OTel) for Go:**
OpenTelemetry is a set of APIs, libraries, agents, and instrumentation to provide observability for applications. You can use the OpenTelemetry Go library to manually instrument your Go Kafka microservices.
- You would need to import the OpenTelemetry Go library into your Go Kafka application.
- Use OpenTelemetry to create and propagate traces as messages flow through your Kafka producers and consumers.
- Ensure that you follow the OpenTelemetry guidelines to set up span contexts and propagate them in Kafka messages.
2. **Custom Instrumentation:**
If OpenTelemetry is too heavy or doesn't fit your needs, you can implement custom instrumentation for your Go Kafka microservices.
- Create custom functions or wrappers around Kafka producer and consumer libraries to record trace data.
- Manually create and propagate trace context in your messages.
- Send trace information as custom headers or metadata within Kafka messages.
3. **Third-party Libraries:**
Some third-party Go libraries and frameworks may offer built-in support for distributed tracing. Check if any of these libraries align with your technology stack and can help with tracing Kafka messages.
4. **Integration with Existing Dynatrace Instrumentation:**
Since you are already using Dynatrace OneAgent for Java, consider ways to integrate Go-based tracing into your existing Dynatrace environment. This may involve custom scripts or integrations to bridge the gap between Go-based tracing and the Dynatrace ecosystem.
5. **Collaboration with Dynatrace Support:**
Reach out to Dynatrace support or your Dynatrace account manager to inquire about any updates or upcoming support for Go Kafka. Dynatrace may have future plans for Go instrumentation, and they can provide guidance or beta solutions.
In any case, regardless of the approach you choose, the goal is to ensure that trace context is correctly propagated within your Kafka messages so that Dynatrace can correlate requests across different microservices, regardless of the language they are implemented in. Be prepared for some development effort to instrument your Go Kafka applications and ensure that trace information is correctly captured and reported to your observability platform.
Thank you, One more question around this, if we have the OneAgent on the Go service with deep monitoring. Can we instrument the Go kafka tracing through the OneAgent detection and transmission or do we have to leverage the Dynatrace OTEL Metrix Exporter method? Since there is a cost in metric units for us to leverage exporters but when using the OneAgent OpenT* Sensor method, there is no cost since the OneAgent is handling the transmission and protocols within the trace.