EF2.0 custom python extension: Task with given ID does not exist

[Mike_Kristofic]

Hi, about a year ago, we have developed a custom python extension on EF 1.0. Now, that we're forced to migrate to EF2.0, we observe some strange behavior.

The code of the extension itself seems to work fine in lab environment, but the extension status seems a bit odd.

If we stop the datasource, after the heartbeat fails, the following log record is observed in python3_fastcheck.log:

[WARNING] api (MainThread): Error HTTP 404 from http://127.0.0.1:14599/events/f_-6912957658320623608: b'{\n "error": {\n "code": 404,\n "message": "Task with given ID does not exist"\n }\n}'

 In Dynatrace logs the well known message appears:

Failed to assign monitoring configuration to ActiveGate. Reason: Fastcheck no response - datasource exited without providing status

Here is a snippet of the extension code around fastcheck definition:

Hope you can give me a nudge in the right direction.

Many thanks!

Mike

[JamesKitson]

The task ID message is usually a result/symptom of the issue rather than the actual issue. It's often associated with that other message about the fastcheck.

"The code of the extension itself seems to work fine in lab environment, but the extension status seems a bit odd." - Does this mean it works running it in a 'dev' mode via the sdk or that it does work as a true extension in some cases? I'm also not sure what you mean by 'stopping' the data source? Do you mean disabling a configuration or something else?

Either way, it's likely related to the failing fastcheck - you should have a python.exe_fastcheck.log file in the datasources directory. For a simple fastcheck like that it's often related to some required module/dependency that's available locally when developing it not being available when it is built and running as an extension due to being missing from setup.py or some other initialization error somewhere in the code - doesn't have to be in the fastcheck section itself, if it can't be initialized at all the fastcheck will fail.

[Mike_Kristofic]

Hi James,

thanks for your reply!

My AG is Linux based, I can only see python3_fastcheck.log logfiles. I suppose the exe one is observed on Windows based AG?

root@agenv-uep:~# ll /var/lib/dynatrace/remotepluginmodule/log/extensions/datasources/custom_ca-extension/
total 48840
drwxrwxr-x 2 dtuserag dtuserag 4096 Jan 8 13:34 ./
drwxrwxr-x 3 dtuserag dtuserag 4096 Sep 5 16:21 ../
-rw------- 1 dtuserag dtuserag 10485872 Jan 8 13:34 python3_fastcheck.1.log
-rw------- 1 dtuserag dtuserag 10485821 Jan 7 17:39 python3_fastcheck.2.log
-rw------- 1 dtuserag dtuserag 10485904 Jan 6 16:54 python3_fastcheck.3.log
-rw------- 1 dtuserag dtuserag 10485762 Dec 16 13:25 python3_fastcheck.4.log
-rw------- 1 dtuserag dtuserag 8020074 Jan 13 14:54 python3_fastcheck.log

Let me be a bit clearer on my setup:

The extension's purpose is to connect to DX Spectrum (formerly CA Spectrum) API, check existing Alerts, compare with Dynatrace Problems and Open/Update/Close the Alerts accordingly.

DX Spectrum in this case acts as the "data source" - that's what I refer to in my original post. If Spectrum is running we can see incoming requests from the extension being processed - that's what I mean by "the exstension seems to be working".

Once we stop CA Spectrum or obstruct the network connection otherwise, the heartbeat will fail and the messages from my OP are observed in fastcheck log and Dynatrace Extensions UI/log events.

There are no records regarding failed dependencies/modules in fastcheck log anymore (we have already resolved a couple during the code migration from EF1.0 to EF2.0).

Let me know if I can provide more information!

Thank you

Mike

[JamesKitson]

Ah, okay. There's an overlap in terminology - we also refer to the Dynatrace piece in the extension framework that runs the collection (e.g. Python WMI, SQL, etc...) a "data source."

The 'fastcheck' log and the corresponding 'task' log should only apply when the extension framework needs to schedule a monitoring task either due to the configuration being updated or an issue with the currently assigned ActiveGate becoming unavailable. Other than that, the fastcheck does not run so I can't see how taking the target of monitoring offline would (or could) be affected by that.

The details of how your extension handles that becoming offline will of course depend on how it was written but nothing I can think of would result in it trying to run a fastcheck unless you're making the original AG unavailable in your test and trigger fastchecks on other AGs. If you are seeing that and can reproduce it, I would suggest opening a support case so the owners of the framework can review it.