21 Mar 2023 10:01 AM - last edited on 12 May 2023 12:32 AM by Michal_Gebacki
I've created a JMX extension for some Kafka metrics but now want to add the connector status metric. According to the confluent link below the connector status can be running, paused, or stopped. Per the JMX Dynatrace documentation, the attributes available in the list are only the numeric ones.
To see other attributes, for example, attributes of the String type, select Include non-numeric attributes. Note that selected attributes must be parsable to double, or boolean type. Otherwise, Dynatrace OneAgent will not collect any data from such a metric. I'm led to believe that the status metric cannot be captured and have seen the same when adding that metric to the JMX extension. Has anyone been able to pull in strings of text such as this status metric?
An alternative way we are thinking of is hitting an API to get the value and then ingesting that into Dynatrace as a metric. However, just want to be sure I'm not missing something with the JMX extension route.
Dynatrace JMX doc
Solved! Go to Solution.
The new JMX extensions in Extension Framework 2.0 will be able to handle this use case. It's currently wrapping up development and will be out in 2 or so months if all goes well. One of the first extensions we are bringing out together with the data source is actually an improved version of the Kafka extension. @victor_balbuena, is connector status one of the new metrics you added?
The connector status is not one of the metrics defined to be captured by the first version of the Kafka extension in EF2.
Thank you @victor_balbuena and @Mike_L. We hope the status is eventually incorporated into the kafka 2.0 extension but in the meantime I can utilize the new jmx framework once available.
Do we know when this kafka connector status metrics will be available ?
The expected date for release of the new EF2 Kafka extension is on the 9th of May.
Thanks for the info. Will this be rolled into a cluster upgrade and be mentioned in release notes? Just wondering how we will be aware of this being available.
It's my understanding that the newer version of the kafka extension will not have the connector status we are looking for BUT we can still create a custom jmx extension to pull out the metric string whereas before EF2 you could not, is that accurate?
It is now available for download from the Dynatrace hub.
you can indeed also create your own JMX 2.0 extensions starting with 1.265.
Could you link in the new documentation for JMX 2.0 for the Dynatrace hub?
I recommend that you grab one of the new extensions, such as ActiveMQ, which has it in the extension.yaml: https://www.dynatrace.com/hub/detail/activemq/?filter=all&type=extension#versions
We have updated to latest dynatrace version and still kafka connector status does not get populated in dynatrace dataexplorer.
To add onto @guruprasadv comment, we have updated the OneAgents to the required version for JMX 2.0 but I can't seem to get the string value to be pulled out for the JMX extension I have created. I looked at (with a dynatrace resource we have) at the ActiveMQ 2.0 extension linked above by @Mike_L and we did not see where that extension is pulling out a string value.
I'm hoping someone has a working example of pulling out a string value using 2.0 JMX. Specifically we are looking for the connector status located at the link below.
Here is an example:
query: "org.apache.activemq:type=Broker" queryFilters: - field: brokerName filter: var:broker_name-filter - field: destinationName filter: var:queue_name-filter dimensions: - key: broker_name value: property:brokerName - key: queue_name value: property:destinationName - key: queue_type value: property:destinationType - key: product_name value: const:ActiveMQ - key: driver_version value: attribute:DriverVersion - key: foo_name value: attribute: Foo accessor: getName() metrics: - key: com.dynatrace.extension.jmx.activemq.queue.const type: gauge value: const:1
As mentioned in my previous comment on this thread, the connector status is not one of the metrics captured by the default Kafka extension. You will have to create an extension of your own. When creating it, take into account that metrics in Dynatrace cannot have a value of a string, which means that I see two possible scenarios you might want to accomplish:
- subgroup: kafka connector status query: kafka.connect:type=connector-metrics,connector=* dimensions: - key: connector value: property:connector - key: status value: attribute:status metrics: - key: com.dynatrace.extension.kafka.status type: gauge value: const:1
Where the dimension is not a
property of the MBean, but an
pausedwould also report a 0):
- subgroup: kafka connector status query: kafka.connect:type=connector-metrics,connector=* dimensions: - key: connector value: property:connector metrics: - key: com.dynatrace.extension.kafka.status type: gauge value: attribute:status accessor: equals("running")
If you want to do something like running=2, paused=1 and stopped=0, that cannot be done at the moment. As mentioned aboved, it is restricted to a boolean so just 0 and 1.
I personally don't see anything else you could do with a string MBean, considering that Dynatrace does not support strings being ingested as the value of the metric.
PD: the dimension for the connector is not necessary, I just added it for more clarity on the example.