We are migrating our data center from West Coast to East Coast. It is a slow process. Currently all the SUD are in the new location (East Coast), and dynaTrace server is in the old location (West Coast). In order to make continue the diagnose, I added a new server for dynaTrace collector and set aside in the SUD network. After the new collector is brought up, dynaTrace server could detect the new collector. The SUD is reboot. However, I still cannot detect SUD from dynaTrace client. Agents Overviews suggests that the agent is disconnected. Instrumentation disabled because of too slow transformation.
I do not believe there is a firewall between SUD and collector. Do I miss anything?
Hi Ronald. Just to double check: The Collector sits in the same network as the Agent - and - they have a good network connectivity to each other? The problem you describe happens if instrumentation of Classes takes very long - this can be explained by two facts:
a) the network connectivity is very bad between Agent and Collector and therefore sending these class files/assemblies takes too long
b) the collector is overloaded and cant instrument these classes fast enough
Can you double check the connectvitiy between Agent and Collector as well as how healthy the Collector is? There is a Collector Health Dashboard accessible from the Start Center as well that gives you overview of the the Collector Health Status
Thanks for the quick respond! Yes, Agents and Collector are in the same network and no firewall. The collector is not overloaded as no traffic is directing to it.
One thing I observed is that, the agent detail still show a default Collector "dynaTrace Collector" when the new collector name is named as "Perf dynaTrace Collector". I have cleared the caches and restart the collector and bounce the SUD. This could be the issue. What else can I do to let the Agent point to the new Collector?
It sounds to me like you have two collectors (or more) in the mix - "dynaTrace Collector" and "Perf dynaTrace Collector" and it also sounds like the Agent in question is incorrectly connected to the "dynaTrace Collector" which might be remote - might even be an embedded Collector. If you go to the dynaTrace Server settings and click on Collectors do you see both collectors listed? Can you check your Agent configuration and verify that it is configured to connect to "Perf dynaTrace Collector" - what is the server= value in the -agentpath for your JVMs (I seem to recall that your app is Java)?
Yes, it is the Agent issue. Originally the Agent in question used embedded Collector. After the data center move, the dynaTrace server was in the original location and all SUD are in the new location. I added a VM as Collector server and called it "Perf dynaTrace Collector".
In order to clarify the host name, please allow me to use dynaServer01 (dynaTrace Server), and dynaServer02 (Collector).
In dynaTrce Server settings, I only see "Perf dynaTrace Collector" (dynaServer02) as my only collector.
The server=value in my Java -agentpath is currently pointing at dynaServer01 (current dynaTrace Server). Does it need to point to dynaServer02?
Yes - the -agentpath must be updated to point to dynaServer02 (=Collector). If you still point it to dynaServer01 (=dynaTraceServer) then the agent is still connecting to the Embedded Collector that runs in your dynaServer01 (=dynaTraceServer).
The "Embedded Collector" should not be used in high load and production environments. So - I recommend two things
1) make sure that all your agents connect to your dynaServer02 (=Collector)
2) turn off the embedded collector in your dynaServer01. You can do this in the Server Settings -> Services -> Uncheck "Allow Agents Connections to dynaTrace Server" -> this will require a dynaTrace Server restart for this change to be applied.
This will make sure that all your agents connect to the Collector. Plus - your dynaTrace Server will have more memory to handle purepaths because it does not need to run an additonal embedded collector in the same process