Are security team is using TCP Sequence Number Randomization on our firewall.
Because of this, we are experiencing high sequence number gap and two-way loss rates. Anyone with a similar setup? How did you resolve your monitoring issues?
Thanks and God bless,
Is the AMD getting packets from both sides of the firewall? If so this will be a problem. Restrict the traffic to one side of the firewall. Packet sequence numbers should be in sequence on one side or the other. Usually I will want the packets on the client side of the firewall so that network performance in that data stream will be reported.
If both streams need to be analyzed the 2 AMDs are required.
In addition to what John said, I can add that sequence randomization alone shouldn't be a problem for the AMD. If AMD does receive packets from both sides, as John suspects, this needs to be corrected first. If this is not the case, then it would be a subject for a Support call - lab will have to analyze packet traces from the AMD to find out exactly what's going on.
Apologies that the diagram wasn't clearer. I am capturing packets from both sides of both firewalls. Actually, there are over 200 firewalls in our network. This is only one application.
Restricting to one side of firewall is not really an option. We need to monitor traffic on both sides. The main issue happens on the from the web servers to the app servers. Traffic from the web server goes through the web tier firewall to the switch to the app tier firewall to the app server. We need to see the RTT, latency, slow ops, etc., and all of the other network pertinent info of traffic between the web and app servers.
I don't follow. In the traffic diagnostics I am seeing between 17% and 49% two-way loss, and between 3% and 15% sequence number gap rates. We have worked with Cisco to fix any SPAN issues. We have worked with Gigamon to fix any packet broker issues. In fact, Gigamon was the vendor who explained the TCP sequence number randomization issue (see attachment - extrahop-tool-reporting-tcp-bytes-missing-from-tcp.docx).
This is just one more time where security enables so much defense in depth (mutliple FW's, F5's, SNAT, TCP sequence number randomization, etc., etc.) that it becomes impossible to monitor the applications on the network. 😞
I'm open to any other suggestions the community may have.
Thanks and God bless,