How to get TIBCO EMS message queue depth?

[ydv999]

Hi Community Members,

We are monitoring of TIBCO EMS and we want to get the details related to the Message Queue depth, Sent Message Size, Received Message Size, etc. but I am unable to find any document stating that.

Could anybody tell me if they faced the same issue and if Dynatrace One agent captures Message Queue Depth and other related matrices?

Please help @Andreas Grabner @Babar Qayyum @Dante Paoloni

Thanks in Advance.

Rishabh Yadav

[larry_roberts · Accepted solution]

I wish I could give better news around this, but it's not possible with the OneAgent and for the life of me I can not figure out why. Tibco is not Java based which might be a big part of it, but it still seems that Dynatrace would see quoue depths and such as important information. We have been after this for the past year. We were told that if we purchased the custom metrics we could build a plugin that could do it, so we built one after finding another one designed for Tibco on Linux. Ours is on Solaris. While technically the plugin works, it is far from ideal. This is just not an area where Dynatrace has matured yet. Here are some of the problems we have found....

• It's not possible to set thresholds within the GUI. You can only do it through code at the plugin level. This alone about makes it useless as we have hundreds of queues owned by numerous teams who do not want to mess with code every single time they want to adjust a threshold.
• The AI does nothing with custom metrics or plugins created even from the JMX Editor leaving us to question the value of licensing custom metrics in the first place as you really can not take advantage of any perks that Dynatrace offers and makes it shine in the first place.
• Compared to what it takes to show such things as Tibco queue depths and metrics based on the JMX Editor, it takes way more code than in other products which I will not mention by name.

Basically it boils down to this... Yes it can be done, but will be all custom and extremely painful to the point that it defeats the purpose of why you would use Dynatrace in the first place because it really does not take advantage of anything within Dynatrace. At the end of the day, you are paying top dollar for custom metrics that you can not really do anything with accept make a basic historical graph which you can do in just about anything.

We have now decided to no longer look to Dynatrace for this and are setting up Prometheus and Grafana to deal with Tibco EMS, JMS, and JMX custom metrics.

I want to be very clear on this - I LOVE Dynatrace and the technology behind it. For the most part it does things VERY well which is why they shine so much. We are actively expanding our deployment of it and I do not see it stopping anytime soon. Queues, JMS, and JMX is just not where they shine and have a long way to go to mature in this area to make it usable in a clean and efficient way. I am not sure how much focus they have on this, but its not from a lack of trying to bring attention to it as we have tried many times over. If you want anything but time series metrics, Dynatrace is the way to go. If you need time series metrics and in a way that is easy to manage, unfortunately Dynatrace is not it. At least not yet anyway.

To be fair, I have been told that some of this might be resolved in Q3, but to what extent and how much of it is the big question.

Just my 2 cents based on our findings over the last year.

[larry_roberts · Accepted solution]

Just an update on this... Dynatrace does now have what is called "Custom events for alerting" in the EAP which I believe is now closed. But with that said, I think it will be heading to GA soon. Only Dynatrace can speak on that. I have done some testing with it, but one bug is still keeping me from doing a real complete test for our needs which would be for Tibco EMS queues. Currently, when you go to configure a custom even and get to the spot where you can select a dimension, the GUI comes back with an "Internal server error" on the Dynatrace side so I never get to a point where I can see the list. Again, this is in the EAP so I would expect to find bugs like this and report them which I did.

There are still some challenges however when trying to use Dynatrace for Tibco EMS queues (requires a custom plugin) IF you have multiple teams responsible for different parts of it.

For example....

If you have one team that is responsible for the Tibco EMS system itself for things such as patching, upgrades, support, etc. and then have individual teams owning individual queues, this is where it becomes a challenge in Dynatrace. Dynatrace from a topology standpoint (and I can understand why) would prefer to place the queues under the Tibco EMS process. This makes perfect sense until it comes time to try and assign ownership which in return leads to notifications, tickets, etc. Because the queues would fall under the Tibco EMS process, this means that any threshold set against a queue would trigger an event for the team who owns the Tibco EMS process, rather than for the queue and / or queues themselves.

The only method I have found to get around this is to have a custom plugin create a new technology and then create a new custom device out of every single queue found. Only at this point can you then tag each queue for ownership because Dynatrace looks at this custom device as a unique device.

I am placing the finishing touches on this ActiveGate plugin. I wanted to create this as a OneAgent plugin, however you can not create custom devices from OneAgent plugins and therefore I have gone the route of the ActiveGate.

As to the bug I referenced above... I have a feeling that internal server error is happening due to the amount of queues it would be trying to pull back. I expect Dynatrace will resolve this otherwise bringing Tibco EMS queues into Dynatrace would be pointless without being able to use the "Custom events for alerting".

I still believe Tibco EMS queue metrics should be out of the box. Dynatrace clearly can see the queues so why not just poll the metrics.

[leon_vanzyl]

Thanks for that Larry, Care to share your Plugin 😉 ?

[larry_roberts]

You're welcome and you bet! I plan to place it out on Github as soon as I have it all completed. I am hoping to be done with it this week. I actually have 2 versions of it.

OneAgent plugin

This plugin will collect queue metrics and place them under the Tibco EMS process and because it is a OneAgent plugin, the AI will utilize the data from it as well. The catch to this plugin is that it's really only good for Tibco EMS users who support everything under one team as you can not set tags on individual queues. It will be a few weeks before this one is completed.

ActiveGate plugin

This plugin will create a new technology called "Tibco EMS Queues" and then create a custom device for each and every queue found. This in return makes each queue unique in Dynatrace and allows tagging of each and every queue for things such as ownership and / or notification. If you have multiple teams that own multiple different parts of Tibco EMS, this would be the best of the 2 to use. The metrics it currently collects are queue depths and sizes.

[leon_vanzyl]

Thanks that would we awesome, I'm working on something similar, I'm hoping to use the hawk api to push some BW Process metrics to dynatrace...

[larry_roberts]

You're welcome. Both of these plugins utilize the Tibco EMS tibemsadmin which provides all the queue metrics. I should note, both of these plugins are designed to be used on Linux and of course will consume custom metric licensing.

[leon_vanzyl]

Yup !! Understood

[josh_wood]

Hi Larry,

Very nice post - thank you for contributing. Did you ever post the OneAgent and/or ActiveGate plugins for EMS to Github? If so, could you share the repo with me?

Best regards,
Josh

[larry_roberts]

Not sure how I missed this and for so long - my apologies. I am going to place it up on GitHub in the coming days. Just need to find the time to get it all around, but I will get it there. Stay tuned...

[alter]

Hi! Any luck with this? Can you share the links?