Latest OneAgent - Possible bug

[larry_roberts]

Curious if anyone else is encountering process crashes on Windows servers related to "oneagentdotnet.dll"?

I have opened a ticket this morning after spending some time debugging to narrow down what was causing some of our applications to fail. I ended up doing a rollback to Dynatrace-OneAgent-Windows-1.219.134 and disabled auto-update on the Windows hosts. After doing so, everything works perfectly.

I have seen this on:

• Windows Windows Server 2016 Standard 1607, ver. 10.0.14393
• Windows Server 2019 Standard 1809, ver. 10.0.17763

Just a heads up!

[AntonioSousa]

@larry_roberts 

I have checked in several environments, and have not one crash related to this in the last 50 days. I do have automatic updates and am running 1.221 in most of them. But I did find several systems with the exact Windows versions that you refer to, without the specified crashes in the last 50 days. Hope this helps!

[larry_roberts]

Thanks @AntonioSousa - From what I can tell on our end, it appears to be specific with IIS. As soon as I downgraded the OneAgents, we have not seen the problem again. I have an open ticket on it, but they are still investigating. 

[ChadTurner]

Thanks @larry_roberts we are planning on upgrading our Oneagent and will take this into account and not past 219. for the time being. 

[apasoquen1]

Thank you for the heads up @larry_roberts. 

[larry_roberts]

You bet! Happy to help 👍

[apasoquen1]

Seems like a 1.221.143 build has been released @larry_roberts.  Have you done any testing with that version?  Would you mind sharing what kind of behaviors with IIS you were seeing with the 1.221.xxx ?

[larry_roberts]

Good morning!

I seen that as well and will be testing it today. Stay tuned...

[ct_27]

@larry_roberts Thank you for posting this.   We have already had two major DT issues with Windows machines (Winpcap/Npcap, and port conflicts) that have taken down 2 critical production applications. I can't afford a third impact so thank you for posting this.

We're in the middle of our monthly patch cycle right now but are stopping it and will skip the 1.221 release.  221 doesn't have anything critical in it and it's safer to just skip it.

Please post if you notice anything else or discover the issue is specific to your company.

-Jason

[larry_roberts · Accepted solution]

You're welcome!
I just tested the newest OneAgent which is "Dynatrace-OneAgent-Windows-1.221.143" and yet again - Process crashes. Reverted back to "Dynatrace-OneAgent-Windows-1.219.162" and the process crash goes away.

Doing some more testing, but support had pointed this out around IIS and asked if I could change the parameters.

So I gave that a try by going into IIS, application pools, and setting "Enable 32-Bit Applications" to "False". As expected, the application no longer functions because it needs this set as "True".
I set it back to "True", rebooted, and the same process crashed. I then took a look at our deep monitoring settings. "OpenTelementry (.NET)" was enabled.

I then disabled this and restarted the server once again and now everything works on the latest OneAgent. So it appears that IIS app pools are not liking OpenTelemetry (.NET) for some reason. 

To be fair, this is marked as "Early adopter".

If you are having issues around .NET, I would highly suggest taking a look at your deep monitoring settings. I am going to update my ticket with this information as well.

Hope this helps!

[ct_27]

@larry_roberts Thanks for taking the time to post an update and with great detail of your analysis.