cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

This product reached the end of support date on March 31, 2021.

DotNet Agent Runtime suspension and i/o time during Load Testing

jerry_lobenstei
Dynatrace Advisor
Dynatrace Advisor

images.docx - screen shots

I do not have a session file as of yet.

We are troubleshooting performance issues with our services
under load. We have found that generally the services are responding quickly
but we have intermittent service calls that execute slowly. We performed load
tests with the dynatrace .net agents both on and off. With the agents off the
response times are much better than when we turn the dynatrace .net agent on.
We still have intermittent slow service calls, but not to the same degree.

With the .net agent turned on, we can see that the slow
service calls seem to be related to suspension (due to garbage collection?).
Though there are subsequent calls that are not flagged as suspension but appear
to be blocked or queued in IIS.

A few questions if you have time.



  • Is
    there anything we can change in the configuration of the .net agent to
    reduce the overhead so the load test numbers are closer with the .net
    agents turned on?

NOTE: We have change the Auto Sensor Settings to Lowest.


  • Is
    there any way to show the time spent on blocked/queued IIS calls? I
    believe this is related to the suspension recorded on the previous calls?

  • The
    % Time in GC counters for the load test are not very high. Are the
    numbers shown above for GC Caused Suspension time above what you would
    consider normal?

  • Is
    there any way to confirm that garbage collection is our issue? Is there
    anything else you can think of we should take a look at?

Application process for duration of the load test images.docx.

4 REPLIES 4

andreas_grabner
Dynatrace Guru
Dynatrace Guru

Hi Jerry

based on the screenshots I think your problem is the long running SQL query. If you have many of those it means you are blocking your ASP.NET worker threads waiting for the SQL to be done. That means that if you have a high volume of requests coming in on IIS these requests eventually get queued on IIS -> which is also what you see in the PurePath that has the high time on IIS. I always also look at Elapsed Time as it shows you where the queuing actually happens. If you see a huge Elapsed Time gap between IIS and the first ASP.NET Node you know that IIS is queuing the incoming request. If you see no elapsed time it means ASP.NET is actually handling it fast but that something takes long on the way back, e.g: large response size, compression, ...

Andi

Hello @Andreas G.

If you see a huge Elapsed Time gap between IIS and the first ASP.NET Node you know that IIS is queuing the incoming request.

So I suppose that ASP.NET
is the bottleneck. What should be done in case we are in this
situation? Do we have to give more resources to .NET Framework? Which?

Thank you!

FYI - here is one of the blogs I wrote where I talk about Elapsed Time issues: https://www.dynatrace.com/blog/c-performance-mistakes-top-problems-solved-december/

I think there are different approaches to tackle this problem

#1: Try to find out whether you can performance optimize your ASP.NET Code. The faster your ASP.NET Worker Threads are done with work the faster they are available to process more incoming requests

#2: You may want to think about scaling IIS. You can configure your IIS AppPools to use Web Gardening. That means that IIS actually launches additional w3wp.exe processes and distributes the load across them. That means you also have more ASP.NET Engines that handle the load

Andi

#1: this is managed from our application Dev team, they do as much as they can.

#2: Yes this could be an option for Ops team, but we must be sure that we have no drawbacks (see https://serverfault.com/questions/81689/web-gardens-are-they-good-or-bad-or-what).

Ops team also added few more frontend machines (load-balanced), but this did not reduced the Elapsed Time in general. So what I'm doing is:

#3: Change machine.config and aspnet.config into .NET Frameworks, in order to give more resources to the CLRs. Do you encourage this change? What parameters do you suggest to change, and to what values?

Thank you