I am noticing really bad latency in our architecture that has a Apache front-end that talks to a Java Application via AJP which in turns requests static content and dynamic content from another java application (Alfresco repository). Right now, i am noticing latency at the point where the Java APP-1 is calling Java APP-2 but can't pinpoint the source of the issue - is the APP-1 choking and unable to send the request to APP-2 or is the APP-2 choking and not responding.
I am seeing high wait times for "executeMethod(org.apache.commons.httpclient.HostConfiguration hostconfig, org.apache.commons.httpclient.HttpMethod method, org.apache.commons.httpclient.HttpState state)"
I am not a Java person and i am struglling to figure out where the issue is. any input/thoughst would be appreciated. I can also attach the fdts file if needed.
The problem most likely has to do with the connection pool size of your application servers. If you make calls from APP1 to APP2 you have two connection/thread pools involved. Those that handle outgoing requests and those that can take incoming requests. In your case you are making calls via AJP - that means you have an AJP connector that can be configured with a certain Thread Size. I remember we ran into the same issue on our Confluence Installation where the AJP Connector on Apache ran with the default setting of 10 Threads. During high load these 10 Threads were not enough. I suggest you google for AJP Connector Thread Pooling. Make sure that this pool is properly sized. then also check the thread pool settings on your Java App Servers.
Thanks for the feedback. here is the architecture
1. request comes into Apache (Apache/2.4.6)
2. Apache forwards request to APP1 via AJP 1.3 (worker.worker1.type=ajp13
APP1 tomcat AJP connector config is shown here
<Connector port="8007" protocol="AJP/1.3" redirectPort="8443" maxThreads="800" connectionTimeout="300000" />
3. APP1 then processes requests and requests data from APP2 via direct http tomcat and these is where i seems to run into a problem. APP2 tomcat's http connector is configured as below
<Connector port="8080" protocol="HTTP/1.1" URIEncoding="UTF-8"
AJP connector for APP2 is configured as below (but i don't think this is used at all)
<Connector port="8009" protocol="AJP/1.3" URIEncoding="UTF-8" redirectPort="8443" />
Could APP1's GC have anything to do with this? I see frequent GC during the load test i am running (10 step random transaction with 150 virtual users for 15 minutes flat)
Thats hard to say. Would be interesting to look at your dynatrace data. I am out of office for the next two weeks. but - maybe somebody else is willing to look at a stored session to identify where the problem really is