topic Re: Problems defining JMX plugin in Jetty / Mule ESB in Extensions

Problems defining JMX plugin in Jetty / Mule ESB

james_billingha — Thu, 05 Jun 2025 09:25:16 GMT

I've got a Mule ESB application that Dynatrace needs to monitor, I see SaaS supports Mule ESB but there are a number of problems. Most pressing is the inability to pick out JMX metrics from Jetty/Mule ESB, which is probably down to me defining the plugin.json incorrectly.

>I have an issue defining the plugin.json, tried multiple ways of accessing the MBeans which unfortunately have "'s and a space in the "name" part of the MBean. Can be seen below for an example app in MuleESB ->

$>beans -d Mule.echo
#domain = Mule.echo:
Mule.echo:Application="application totals",type=org.mule.Statistics
Mule.echo:Flow="EchoFlow",type=org.mule.Statistics
Mule.echo:connector=connector.http.mule.default,name="endpoint.http.localhost.8084",service="EchoFlow",type=Endpoint
Mule.echo:name="EchoFlow",type=Flow
Mule.echo:name="_muleSystemModel(seda)",type=Model
Mule.echo:name="application totals",type=Application
Mule.echo:name="connector.http.mule.default",type=Connector
Mule.echo:name=AllStatistics,type=Statistics
Mule.echo:name=Configuration
Mule.echo:name=MuleContext
$>get -d Mule.echo -b Mule.echo:Application="application\ totals",type=org.mule.Statistics TotalProcessingTime
#mbean = Mule.echo:Application="application totals",type=org.mule.Statistics:
TotalProcessingTime = 5290;

My plugin.json looks like (Simple one to get just one value as a test) ->

{
  "version" : "0.001",
  "name" : "custom.jmx.muleesb",
  "type" : "JMX",
  "entity": "PROCESS_GROUP_INSTANCE",
  "configUI" : {
		"displayName": "MuleESB JMX"
	},
  "processTypes": [ 10, 12, 13, 16, 17, 18 ], 
  "metrics" : [
    {
      "timeseries" :
      {
        "key" : "TotalProcessingTime",
        "unit" : "Count",
        "dimensions" : [
          "rx_pid"
        ],
	"displayname": "Total Processing Time"
      },
      "source" : {
        "domain": "Mule.echo",
        "keyProperties": {
	  "type": "org.mule.Statistics",
	  "name": "application totals"
        },
        "allowAdditionalKeys": false,
        "attribute": "TotalProcessingTime",
        "calculateDelta": false,
        "calculateRate": false,
        "aggregation": "SUM"
      }
    }
  ],
  "ui" :
  {
    "keymetrics" : [
      {
        "key" : "TotalProcessingTime",
        "aggregation" : "avg",
        "mergeaggregation" : "sum",
        "displayname" : "Total Processing Time"
      }
    ],
    "keycharts" : [
      {
        "title" : "MuleESB Total Processing Time",
		"group" : "MuleESB metrics",
        "series" : [
          {
            "key" : "TotalProcessingTime",
            "displayname" : "Processing Time",
            "rightaxis" : true,
            "color" : "rgba(101,152,186,0.4)",
            "seriestype" : "bar",
            "mergeaggregation" : "sum"
          }
        ]
      }
    ],
    "charts" : [
      {
        "group" : "MuleESB metrics",
        "title" : "MuleESB Total Processing Time",
        "series" : [
          {
            "key" : "TotalProcessingTime",
            "displayname" : "Processing Time",
            "rightaxis" : true,
            "color" : "rgba(101,152,186,0.4)",
            "seriestype" : "bar"
          }
        ]
      }
    ]
  }
}

The issue seems to be with "name", it should be "\"application\ totals\"", but Dynatrace won't accept any escaped characters like that. I also tried using "name": "*" and that also fails ...

I say "fails", unfortunately unless I do something like put in "\\\"application\ totals\\\" and get an exception I get nothing in the logs at all for the monitored process. But what I do see is ->

2018-02-28 14:20:46.745 UTC [7e19f84a] info    [native] Updating plugins content from revision 1519827060897 to 1519827091624 (11 plugins)
2018-02-28 14:20:46.745 UTC [7e19f84a] info    [native]   Adding plugin custom.jmx.muleesb with version 0.001
2018-02-28 14:21:16.187 UTC [7e19f84a] info    [native] Updating plugins content from revision 1519827091624 to 1519827060897 (10 plugins)
2018-02-28 14:21:16.187 UTC [7e19f84a] info    [native]   Removing plugin custom.jmx.muleesb
2018-02-28 14:21:26.547 UTC [7e19f84a] info    [native] Updating plugins content from revision 1519827060897 to 1519827091624 (11 plugins)
2018-02-28 14:21:26.547 UTC [7e19f84a] info    [native]   Adding plugin custom.jmx.muleesb with version 0.001

It "adds" and "removes" the plugin over and over ...

I've got the plugin.json above working for a more conventionally named MBean in the same process. So "get -d Mule.echo -b Mule.echo:name=MuleContext FreeMemory" works in the above and I get a chart with free memory in it on the UI showing the rest of the config is correct. It just can't read the "application totals" name key as far as I can see.

Any ideas? I've tried escaping, using *, using unicode.... Nothing works and there is no debugging info!

Re: Problems defining JMX plugin in Jetty / Mule ESB

Julius_Loman — Wed, 28 Feb 2018 19:34:18 GMT

Did you try to use the JMX Browser when defining plugin? See https://www.dynatrace.com/news/blog/jmx-plugins-editor-now-available-eap/

Re: Problems defining JMX plugin in Jetty / Mule ESB

james_billingha — Wed, 28 Feb 2018 22:36:41 GMT

Cheers, I've asked, but I assume that is a nice UI for defining the plugin that ultimately executes on the OneAgent the same as any other. It is there that the execution fails as it can't read an MBean with that name (I assume). But I'll give it a go, looks a lot more user friendly!

Re: Problems defining JMX plugin in Jetty / Mule ESB

Julius_Loman — Thu, 01 Mar 2018 07:47:01 GMT

It is definitely worth trying it. If you can't see the mbean, did you look also in the agent java logs for the monitored process? For JMX plugins data will not be in the Oneagent plugin logs, but in the Oneagent Java logs, since it's a java process and JMX attributes retrieval is performed using the Java instrumentation. Oneagent plugin logs are for the Oneagent SDK plugins only.

Re: Problems defining JMX plugin in Jetty / Mule ESB

james_billingha — Thu, 01 Mar 2018 12:31:34 GMT

Log I found the messages above in was in /opt/dynatrace/oneagent/log/java, named for the MuleESB process - ruxitagent_org.mule.module.reboot.MuleContainerBootstrap_16962.0.log

I assume that's the right one?

Thanks again!

Re: Problems defining JMX plugin in Jetty / Mule ESB

Julius_Loman — Thu, 01 Mar 2018 15:45:36 GMT

That should be the correct one. Unfortunately for increasing debug log you need to open a support ticket.

Re: Problems defining JMX plugin in Jetty / Mule ESB

james_billingha — Fri, 09 Mar 2018 13:53:14 GMT

@Julius's solution worked, the plugin.json it created had the quotes delimited but not the space. Thought I'd tried that combination but apparently not!

Re: Problems defining JMX plugin in Jetty / Mule ESB

james_billingha — Fri, 06 Apr 2018 11:23:06 GMT

Might be useful for others instrumenting MuleESB JMX, given you need a JMX config per application this script auto-generates one for every application defined - https://github.com/JD-Billingham/Dynatrace_MuleESB_JMX

Re: Problems defining JMX plugin in Jetty / Mule ESB

jeppe_lindberg3 — Fri, 02 Nov 2018 09:41:50 GMT

@Michael L. this is worth looking into