While running Stress test we saw servers reaching 100% cpu. And when doing CPU sampling it shows dynatrace instrumentation CPU overhead is high and thread dump show that it has 29 blocked threads by Dynatrace.diagnostics.agent.af method. Any idea about these issues?
This can likely be caused by a misconfiguration. I would suggest removing any custom instrumentation, setting JDBC to Aggregation, logging sensor to aggregation, Exception sensor to not capture stack, ensuring String capture length is small (256?), then retesting. If it still happens, remove sensors for JDBC, Logging, Exception. The goal is to determine which sensor is causing the problem. You might be able to get some hints about which sensor is involved from the thread dump you mentioned, look at the stack above the dynatrace agent. JDBC? Once you determine the offending sensor or setting, you can adjust accordingly.
Is it possible to create a dashboard with any measure for all the agents in the system to identify dynatrace CPU instrumentation Overhead in Appmon v7.0?
last time we identified it when we did sampling for few servers. as we are doing stress test CPU will be 100% and would like to see if any of the agents has any overhead caused by Appmon.