cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

RabbitMQ Metrics Extension management/connection per Host

steven_vue
Participant

Hello,

 

We are utilizing the RabbitMQ (RMQ) Metrics Extension plugin in our SaaS environment for PCF. We have the RMQ nodes instrumented with Dynatrace and have access to those out-of-the-box metrics. We noticed that when we enable the RMQ Metrics Extension plugin (under Monitored Technologies) per RMQ node, we tend to lose the connection (or it looks like the connection is lost):

steven_vue_0-1641397524510.png

 

Is there any way we can:

1) monitor the status of the RMQ Metrics Extension on EACH RMQ node

2) automatically re-enable the RMQ Metrics Extension on EACH RMQ node

3) How to troubleshoot why the extension gets disabled/lost/disconnected

 

We are not enabling or plan on enabling RMQ Metrics Extension at the Global level to avoid impact to any RMQ services and/or apps using the services. We are practicing an "opt-in" model hence enabling the extension per RMQ NODE/Host. 

 

Hope that makes sense!

 

Thanks,

Steven

8 REPLIES 8

bradley_danyo
Dynatrace Helper
Dynatrace Helper

Hi Steven,

 

If I'm understanding your question correctly, you should be able to click on the use host configuration button from your screenshot to monitor each node.  You could also look at the extensions API for some automation.  Hopefully this helps.


Regards

steven_vue
Participant

Hi Bradley,

 

So we do that and get the extended metrics, but for some reason, after our RMQ software version is upgraded, this is what we see when we open the RMQ Host Settings --> Monitored Technologies --> RabbitMQ

 

bradley_danyo
Dynatrace Helper
Dynatrace Helper

What version of RMQ are you using?  Also, what does it look like in the global settings?  If you go to Settings --> Monitored Technologies --> RabbitMQ?  It looks like you're using the global settings rather than the individual host settings.

steven_vue
Participant

RMQ version: https://docs.pivotal.io/rabbitmq-cf/2-0/releases.html#2-0-6

 

Settings --> Monitored Technologies --> RMQ : 

steven_vue_0-1641402933386.png

 

you'll notice that we do NOT have it enabled at the Global Level Settings (hence the "250 RMQ hosts need configuration"). 

The screenshot earlier was from an individual RMQ host that we did have enabled at some point but no longer is connected. You can see that I am in the RMQ Host settings itself.

steven_vue_1-1641403261818.png

 

bradley_danyo
Dynatrace Helper
Dynatrace Helper

Ok thanks for the info.  It looks like you'll need to select the use host configuration button from your second screenshot and fill out the info below (user, password, port, etc.).  Can you show me what the RabbitMQ process running on that host looks like in Dynatrace?  Just a screenshot of the process overview page like this: 

bradley_danyo_0-1641405590669.png

 

 

steven_vue
Participant

So we entered the creds using the Host --> monitoring overview --> RMQ Extension

Here's the process page:

steven_vue_0-1648570361431.png

 

bradley_danyo
Dynatrace Helper
Dynatrace Helper

Are these containerized?  There may be an extra level of configuration needed if the OneAgent is also running within a container but  its just a matter of setting up the necessary rules so the networking can go from OneAgentPlugin process within OneAgent container --Out To Host--> routing rule on host --back into app container--> Access to RabbitMQ.  You could try setting up some kind of routing table rule on the host to redirect from localhost:<anyPort#> to the specific IP and port of the container that is running RabbitMQ.

Hi Bradley,

 

Since we are running in a virtual CF environment, these RabbitMQ VMs are often recreated and destroyed and do not come back up with the same VM-ID or IP, hence Dynatrace is not able to re-enable the Dynatrace RabbitMQ Extension. 

 

Normally the same IP address is re-assigned to the same VM (including a replacement VM).

 

However, recently there has been technology enhancement called "BOSH Hot Swap VM" where a new VM is created while the old VM is still running then once the new one is online a quick network re-routing take place to send traffic to the new VM before the old is deleted. In that special scenario then the IP address change because the new VM has to have its own IP address to make this works and therefore cannot be assigned the old IP.