As you can see in the image an api call comes in, it does a simple operation then an asynchronous operation starts and 4 seconds later it executes.
I'm trying to determine why the 4 second delay. This is .Net code on a simple mvc controller and we're not controlling any of the synchronous & asynchronous aspects of it.
I'm trying to determine what is going on, was IIS just busy?
This does not happen on every request but out of 30 there are about 5-10 of them
Could it be because of "hand request" in IIS ? (Saturation on the number of IIS). We already have the case but it was between the IIS and ASP.NET part.
I am very interested if you find why.
Whats the TicketManager class doing? Is this part of the regular ASP.NET Request Processing or is this part of a custom IIS Module that e.g: does authentication? I assume the latter which means that IIS first calls the different registered IIS Modules and later passes the request to the ASP.NET Runtime for regular request processing. The time difference could be explained by either a module that gets executed after the TicketManager module and it simply takes time. It could also be that you are not having enough ASP.NET Worker threads available which means the request itself queues before it actually gets executed by ASP.nET
The TicketManager is authenticating the request, its our code that we make available to IIS when it needs to authenticate a request.
How would I determine if this is another module or not enough worker threads/
You can monitor the number of worker threads at the time when you hit this performance issue. Dynatrace will automatically monitor these threads. If that is not conclusive I typically recommend to use FRT (=Failed Request Tracing) which is a feature from IIS which gives you a detailed breakdown of a request into each module execution.
We are currently working on a built-in feature that will deliver that type of insight out-of-the-box. until we have this I suggest you try FRT