I'm planning upgrade my existing 12.3.2 to 12.4 and concerning about the backward compatibility as well as new AMD throughput. Can anyone give me some advises? Thanks in advance.
I was told the CPU dispatcher processes on 12.3.2 will be improved in 12.4, any information mentioned the throughput of new AMD? I have 15 GB/s from a total of 6 SPAN port after packet de-duplication. I wonder if 2 x 10GB can handle it all.
On the other hand, is DCRUM 12.4 backward compatible with:
TCAM (already found the answer from another forum thread)
Any ideas are welcome.
Solved! Go to Solution.
The capabilities of a 12.4 AMD are roughly the same as in 12.3. However the High Speed AMD that will be available in 12.4.5 has much greater capacity.
The main limit of the AMD's performance is not the throughput spanned to the AMD (as this is mainly limited just by the NIC hardware capabilities), but the amount of traffic actually analyzed (i.e. traffic belonging to user defined software services or all traffic in case of Autodiscovery).
The safe 12.3 limit tops at around 300kpps (300 thousand packets per second), depending on the analyzed protocols. In your case, 15 gigabit per second equals to around 3 million packets per second - so the AMD would be capable of analyzing at most 10% of the traffic seen on the NICs (the rest would need to be filtered by means of configuration).
Thanks for your reply. I'm planning to migrate the AMD hardware as well as 12.4 to cater more SPAN ports traffic so want to estimate the AMD capacity before order the hardware and NIC.
Currently my AMD is using 2 x 10GB SPAN port to collect traffic, would you help suggest what DMI metrics help estimate the capacity of AMD? Does total packets/sec, realized bandwidth and total bandwidth usage help? Or something else?
Thx a lot.
Those metrics are good starters, I'd also look to the individual CPU utilisation on the AMD. AS you are aware pre 12.4 the dispatcher process would run out of CPU (look for 1 CPU at 100% all the time, usually CPU0 but not always) and become a bottleneck before the analysis engines did.
The Diagnostics -> AMD Statistics -> Individual AMD Packet Statistics -> Packet distribution (analysis level) -> analyzer_Analyzed_packets metric is a good indication of the AMD load. When viewing the report in the default 1-hour resolution you can divide the displayed value by 3600 to get the analyzed pps value. Looking at one period resolution reports is also useful, as it shows short traffic peaks.
Thanks Chris and Wojciech.
From the AMD analyzer_Analyzed_packets (1 period resolution), the packet peak at 15.8 M, if I understand correctly i.e. 15800000 packets / 5 min / 60 sec = around 53 kilopackets.
If AMD tops at 300 kpps/sec, then it means the AMD still have plenty of room?
To be honest, from section "Packet distribution (driver level) sometimes I found "Dropped_packets_sampling", I pick one interval as sample below. However, compare with the POC conducted last month which had Gigamon in-between the SPAN port and AMD, the dropped ratio were greatly reduced.
I really hope to see the AMD can work as what a local representative told me "AMD can handle 20G of traffic in 12.4" 😉
Your theory is right Sylvian, but the sample rate for the AMD stats is 1 minute (not 5). So we're looking at ~260kpps
is nearing the 300Kpps 'limit', there's no hard and fast limit,
there's too many variables to consider to provide a bandwidth or packet
limit for an AMD processing capacity.
The 20Gbps capable AMD, is currently in beta, not due til 12.4.5 (April, not but set in stone as product schedules can change). The required hardware specifications for such an AMD have yet to be released, but assume they will be significantly greater than current AMD.
Chris, You're right. I forgot the resolution of AMD stat report is 1 minute.
I'm using Auto Discovery so can't get rid of those "Other TCP" and "Other UDP", I believe if I could filter out them before RTM process, the kpps should be decreased. ^_^
For the Next Gen AMD, although not official published, the specs required for the HW will have as a minimum requirement, newer dual Xeon CPU's with at least 20 cores combined, 128GB of RAM, and 15K SAS drives in a RAID 10 configuration. Anything above this will futureproof your AMD as well (especially cores and RAM, as 16 core CPU's aren't that much more expensive, and extra RAM is always beneficial). By the way, this all fits very nicely in a 1RU form factor, even with mutliple NIC's.
However, one important point to clarify, is that as of now, the High Speed AMD (AMD NG) will require a different license. The standard licenses (includes existing AMD licenses on maintenance) will not be rated to handle 20Gbps, but will be rated at a lower level (TBD, but a bit above what a well provisioned AMD in 12.3 can handle for TCP analysis). Although you can install your AMD on the new HW, it won't be licensed for 20Gbps, that will require an AMD NG license. Even though that may sound low, the other thing to look at is that on newer HW specs, that traffic level won't be for just basic TCP analysis (as it is in 12.3), but you'll be able to run decoded traffic through there at near the same pace, as opposed to the impact that decoding traffic now has on the rate that 12.3 AMD's can handle.
Your local Rep and SE can provide more clarification.
Thanks Michael and Jaroslaw.
Michael, Great reminder on "New AMD NG license". Let's say AMD NG, 20 Gbps means 2 X 10 Gbp SPF+ card is fair enough (I believe). The sample High Speed AMD in documentation is a Dell but I prefer CISCO UCS since it is corp standard.