IBM MQ extension includes cpython linux .so file?

[AntonioSousa]

While debugging an error I'm having with the IBM MQ extension, I have noticed that it includes in the extension ibm_mq-3.9.5-py3-none-any.whl, the following binary files, inside the ibm_mq/pymqi directory:

• pymqe.cp310-win_amd64.pyd
• pymqe.cpython-310-x86_64-linux-gnu.so

As I believe I have not seen this before, was wondering (as an Extension developer) why it was done this way? Some optimization trick? Because I believe it would have been setup better as a setup.py dependency, with the corresponding .whl file in the extension. Or ist it a dependency from the other jar file?@Mike_L , can someone from you team give a hint?

[diego_morales · Accepted solution]

Hi @AntonioSousa 

It cannot be added in setup.py because it has to be built using the IBM MQ SDK local to the machine. That .pyd and .so files look for the IBM MQ SDK libraries on the machine it is built, then compiles its CPython using those libraries.

They need to be pre-built on each OS and then added manually for that reason. It only executes the one it recognizes by the OS when the extension runs.

Hope this answers your question.

Thanks,

Diego

[AntonioSousa]

@diego_morales ,

Thanks for the explanation.

I was aware of the native dependencies, but had the idea they were distributed as whl's. That's at least what we usually see with libraries like cryptography, which are included in separate linux and windows whl's. I believe this ibm_mq is really the only extension where I have seen the binaries inside the "main" whl of the extension.

[diego_morales]

Indeed, they usually are. These ones require that combination of native MQ SDK libraries to compile, so a simple .whl won't do. That extension is also unique in many ways considering IBM made MQ very customizable in different OS and distributions and they all behave different. The extension had to accommodate for all that as well.

[jiri_stefanek]

Hi @diego_morales 

I have a question regarding the IBM MQ extension. We have this extension running in a Managed environment without any issues, but we recently migrated to SaaS (the configuration is identical to the Managed setup; the only new component is the ActiveGate in SaaS). Unfortunately, in SaaS we are getting these errors. The firewall on the new ActiveGate is open for the same ports as in the Managed environment.

Do you have any idea what might be causing the issue or what we should focus on?
What’s strange is that besides errors and warnings, the status also shows “OK”, but we still don’t receive any data.

Warn

Content

Endpoints OK: 0 WARNING: 0 ERROR: 1 Unhealthy endpoints: xxxx - GENERIC_ERROR DEC:15C ERROR: Unable to connect to queue manager xxxxx. Could not connect to Queue Manager: Code: MQCC_FAILED, Reason: MQRC_HOST_NOT_AVAILABLE. Message: MQJE001: Completion Code '2', Reason '2538'.

Error
Content

TIATUOAQM: [NEW] - GENERIC_ERROR DEC:15C ERROR: Unable to connect to queue manager TIATUOAQM. Could not connect to Queue Manager: Code: MQCC_FAILED, Reason: MQRC_HOST_NOT_AVAILABLE. Message: MQJE001: Completion Code '2', Reason '2538'.

Info

Content

Endpoints OK: 1 WARNING: 0 ERROR: 0
Info

Content

xxxxx: [NEW] - OK

The only thing I noticed is that Managed has for examle in event logs
dt.entity.extensions:extension
CUSTOM_DEVICE-xxxxxxxxxxx
but SaaS has
dt.openpipeline.pipelines
system.events:default

[Julius_Loman]

@jiri_stefanek your error clearly states the extension cannot connect to MQ manager. Most missing firewall clearances between AG and MQ (if you deployed a new AG for SaaS). Or a different configuration.