The following tips are related to Wireshark traffic analysis.

1. How to decrypt SSL traffic using Wireshark?


You may want to decrypt SSL encrypted traffic to examine traffic details. If AMD is not decrypting 100% of the SSL traffic, Wireshark can be used.

  1. Open Wireshark from the computer's Start or Application menu.
  2. Click "Preferences" on the Wireshark toolbar.
  3. Click "SSL" on the left pane of the Wireshark Preferences window.
  4. Place a check mark next to "Reassemble SSL records spanning multiple TCP segments" and "Reassemble SSL Application Data spanning Multiple SSL records."
  5. Enter the IP address and port of the server in the RSA keys list field. Follow this with the protocol to interpret decrypted values as, then finally the location of the private key. Separate each value with a comma. For example, if the server is running on "" and operating on port "443" over an encrypted "http" protocol and the key is in "C:\temp\privkey.pem," you would enter the following in the "RSA keys list" field:,443,http,C:\temp\privkey.pem
  6. Enter a location for a debug log file to be created on the hard drive, if you would like to generate one.
  7. Click "OK." Wireshark will now capture SSL records on communications with the server and automatically decrypt them.


Root cause analysis for missing SSL-related operations in CAS

Currently, the AMDs are decrypting between 85 and 100% throughout the day. The 64bit AMDs were decrypting 93-100% of the traffic when the traffic volume was limited. Mostly, the AMDs decrypt and report 100% of the operations/traffic except for various isolated times (i.e. 1hr mid-morning). All operations not reported in the CAS have been identified to be traffic issues causing problems with decryption. These fall into the following scenarios from most to least often seen:

  • IP Identifier = '0'
  • Excess traffic sent to the AMD
  • Packets missing in traffic
  • Session without data ending in a client reset
  • Duplicate traffic

In order for DC RUM to decrypt the remaining 0-15%, these traffic issues need to be addressed from a network perspective. Each scenario is described below.

A) IP Identifier = "0".
During the handshake, IP ID is used to de-duplicate traffic. De-duplication fails resulting in the failure of decryption for the entire session.
Possible resolutions are to reduce duplicate traffic or to change configuration of de-duplication setting to SYN-ACK. There may be traffic scenarios which may not be handled by SYN-ACK.

Packet without an ID:

Packet with an ID:

B) Excess Traffic that results in AMD performance.
The AMD is analyzing very little of the traffic (5-20%) being sent to it. This can cause the custom driver to get overloaded filtering packets and unable to send all the packets to the analysis engine. This is shown below as rx overrun:. A more dire scenario would cause the driver to drop packets.

C) Actual packet traffic is not sent by Gigamon/Spanning device.
We reviewed a packet capture from the AMD during the time when CAS reported fewer operations than the web server log. In the captured packet, loss was reported at 8:37. We compared this capture with the one from OPNET to determine if the AMD was dropping the packet or the packet wasn't forwarded from the Gigamon. Since both captures were missing the same packet, it stands to reason the Gigamon did not forward the packet. A possible cause is an overload of network equipment. This loss of a packet causes the AMD to be unable to decrypt any operations in progress or any after the packet is lost.

In Wireshark, the lost packet is shown in the following capture with the packet info '[TCP Previous segment lost]'. You can see in the both captures below a packet with info '[TCP Previous segment lost]' where both have the same 'Sequence number'. This denotes that in both captures the exact same packet was not seen in the capture.

AMD capture:

OPNET capture:

You can see in the below screenshot the AMD reporting a packet is lost from a client and a server in the rtm.log file while in SSL debug mode.

Another scenario where packet loss is seen is with the loss of a retransmitted packet. Below you can see the message '[TCP Previous Segment lost]' where the lost packet was retransmitted in the next packet '[TCP Retransmission'. This will also result in a failure to decrypt remaining operations.

D) Sessions that contained no data and ended with a reset packet from the client.
This generates a PrematureSessionEnd error on in the SSL decryption statistics on the AMD. This isn't an error in decryption as we processed the traffic correctly. The error simply indicates this scenario. You can see below a SYN packet immediately followed by a 'RST, ACK' then later another 2 'RST' packets.

E) Excess duplicate traffic.
The AMD has limited buffers to handle duplicate packets. If enough duplicate traffic is seen at once the buffer can fill the AMD can then encounter problems decrypting the traffic as the necessary information would not be available for analysis. In the capture example below you can see two '[SYN]', '[SYN, ACK]' and '[SYN]' packets each pair with the same information.

F) Traffic is not seen from the beginning.
This generally happens when the AMD has been restarted. Once an AMD is restarted a new SSL handshake needs to be seen to decrypt the traffic.

2. How to open large packet traces in Wireshark?


Even if you have 4GB of RAM on your workstation you may encounter problems with opening large packet traces (say 2 GBs large or bigger). Wireshark will terminate with out of memory exception like this:


Here is what you can try:

1) apply packet filtering rule(s) while opening the file:

The syntax is the same as in Wireshark.

2) Start Wireshark.
a) Chose '
View -> Colouring rules' in main menu, select all rules (CTRL+A) and click 'Disable'
b) Choose "Edit Preferences->Protocols" and turn OFF the two config options :
for "IP" protocol disable 'Reassemble fragmented IP datagrams'
for "TCP" disable 'Allow subdissectors to reassemble TCP streams'

These settings can be stored under a separate configuration profile in Wireshark, if you don't want to change your defaults (Edit -> Configuration Profiles).




  • No labels