Data Center RUM Documentation

Skip to end of metadata
Go to start of metadata

Choose one of the available search methods to extract datamining entities in the selected search scope.

Each search method requires you to specify a different set of extraction rules.

Add prefix

Use this method if you expect the value to always be preceded by a specific prefix. To extract the value, provide the prefix expected to precede the value.

Cookie name search

Specify the cookie from which to extract the value. Provide the value of a specific cookie name confirming a successful login. The session ID, for mapping to the value, is extracted from this cookie. Successful logins are normally recognized by a SET COOKIE operation for the named cookie

Decode / decompress

If you expect to perform a search on a compressed or encoded data, or URL encoded in case of URL parameters, you can bring the search results to a human readable form by using one of available decoders, Base64, Base64 + Gzip, Gzip or URL encoding.

You can also extract parts of your initial search results by using Text search or Regular expression search methods. Choosing method of searching within the payload

MQ header search

Use this option to extract a specific MQ field from the MQ communication. Next, select the MQ field to be extracted.

Mime encoded list filter

Use this method if you expect to find a value in an MIME format. Provided values, if found, will be filtered out. Including text in character sets other than ASCII, message bodies with multiple parts and in header information encoded in non-ASCII character sets.

Nth element search
Use this option to extract Nth parameter from input using delimiting character. You can set ordinal number telling which parameter to extract. Zero means last. Set delimiter which is used to split input into separate parameters.

NTLM search

Use this method to search for a value in an NTLM authentication request header. Depending on your choice, the value can be composed of the following fields: workstation, domain, or user. Select the fields that compose an identified value and, if necessary, change the default character used to separate the selected components in the resulting value. Note that AMD supports NTLM NTCR - NT Challenge/Response authorization. SPNEGO-based kerberos authentication is not supported.

Parameter name and value search

Use this method if you expect the value to always be carried by the specific parameter. To extract the value, provide the parameter name. Depending on the selected search scope, the term parameter may refer to a specific entity, such as a cookie name (when the search scope is set to cookie), or a header field (when the search scope is set to request or response header).

Parameter name prefix search

Use this method if you expect the value to always be carried by a specific parameter with a specific prefix. To extract the value, provide the parameter name prefix and indicate what data should be reported. The results of the search can be presented as a parameter name and the value, just the parameter value or just a parameter prefix.

Parameter value suffix search

Use this method if you expect the user name to always be carried by the specific value of a parameter with a specific suffix. To extract the user name, provide the value for the suffix.

Regex search

You construct a regular expression that, when applied to a selected search scope, returns the value. The regular expression must contain at least one group enclosed in parentheses. If the regular expression returns a number of search groups, you can define the custom group order by entering a comma-separated list in the order of your choice (for example, 2,1,3 ). This method is not available for the cookie and response body search scopes. For more information, see Regular expression fundamentals .

You can test the patterns that will be used by the AMD using the Regular Expressions Test tool, which is activated after you click Test located next to the regular expression pattern field.

Figure 1.

The following is an example of extracting the value of REMOTE_ADDR field from the HTTP header.

An HTTP header might contain the following information:

        GET http://www.slow-server.com/login.jsp HTTP/1.1
Accept: */*
Referer: http://www.slow-server.com/
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
Host: www.slow-server.com
Connection: Keep-Alive
Cookie: FPB=061j8hura11q56cv; CRZY9=t=1;
REMOTE_ADDR: 10.1.0.2
      

The following regular expression extracts the address 10.1.0.2 from the REMOTE_ADDR field:

The expression must contain a single sub-expression delimited by pairs of characters \( and \). The expression in this example states that the search string should start at the beginning of a header line and end at the end of the line (note the use of % to denote the hex values of the carriage return and line feed characters). The line should start with the string REMOTE_ADDR:. The sub-expression to extract is a string of characters different than ASCII CR or LF, and it should occur after the space following REMOTE_ADDR:

Regex with replace search

Use this method to construct a regular expression which when applied to the selected search scope, returns a value. Use POSIX extended regex syntax that will be matched against the input text.

You can apply the regex to all occurances, By default, the regex will be applied only once and stop after the first substitution.

You can also include the unmatched content, Otherwise, only the text matched using the regex will be included in the output.

Text phrase search

Use this method if you expect the user name to always be found in the text . The provided value for the search parameter will be used to match the text phrases in the analyzed traffic.

Text search

Use this method if you expect to find a user name between the first occurrences of strings defined by Match start and Match end. Because it is not always possible to extract the user names directly, you can use this method as a first step in preparing content for search result transformations. You can set a Search limit in bytes to avoid lengthy search results. This method is not available for the cookie search scope.

XML attribute search

Use this option to find a certain XML attribute within a certain XML tag and then return either the tag name or the attribute value.

Tag nameThe XML tag name within which you want to find a certain attribute.
Attribute nameThe XML attribute you want to find.
Search limitThe number of bytes to search within a file (to limit possibly large searches). The default should suffice in most cases.
Occurrence

Which occurrence of this match (attribute within a tag) you want to return. For example, if you want to find the third occurrence of this match within a file, set Occurrence to 3.

  • Select Any to match any of the occurrences of the defined search step. For example, the given extraction step set as Any will attempt to match the definition of the next step. Only until all subsequent steps are matched the search step set to Any will be considered matched. See an example.
  • Select Count sibling occurrences only to count only the occurrences that happen at the same level within the XML structure, and to ignore any occurrences at other levels within the XML structure.

  • Clear Count sibling occurrences only to count all occurrences regardless of their place in the XML structure.

ReportWhether to return the Name, Value or Any. The Any option will report either the name or the value (whichever is present), and it is the best option to use if you are not certain what the observed XML traffic consists and you wish to avoid blank reports.

You can set the Occurrence to Any which will allow you to create search dependencies that will match the Any step only when all other steps are matched.

 Any occurence example that returns a value of an attribute.
Returns a value of a specific attribute.

Steps Configuration:

  1. XML body search

    Tag namePART
    Search limit4096
    Occurrence1
    Count sibling occurrences onlyunchecked
    ReportBranch
  2. XML attribute search

    Tag nametagname2
    Attribute nameattrname
    Search limit4096
    Occurrence1
    Count sibling occurrences onlyunchecked
    ReportValue

Observed traffic:

Search result:

The result is unmatched since, the tagname2 occurs in the second <PART> tag and XML body search is configured to examine only the first occurrence of the <PART> tag.

Changing the XML body search occurrence to Any allows the search to attempt to find the tagname2 in any of the <PART> tags observed in the XML traffic giving the search result from the second <PART> tag:

Icon

Be aware that, a step defined with Any option is will be considered matched only when all subsequent steps are matched.

You can leave the Tag name or Attribute name fields empty which will affect the search results.

 XML attribute example that returns a value of a specific tag.
Returns a value of a specific tag.

Leaving the attribute name blank will return a value of a specific tag:

Configuration:

Tag nametagname
Attribute name 
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportValue

Observed traffic:

Search result:

 XML Attribute example that returns a name of searched attribute of a specific tag.
Returns a name of searched attribute of a specific tag.

Useful if you want to check if a particular attribute is present in a particular tag.

Configuration:

Tag nametagname
Attribute nameattrname
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportName

Observed traffic:

Search result:

 XML Attribute example that returns the name of a tag.
Returns the name of a tag.

Useful if you want to check if a particular tag is present in a document.

Configuration:

Tag nametagname
Attribute name 
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportName

Observed traffic:

Search result:

 XML Attribute example that returns the name of a first tag found (required for SOAP traffic).
Returns the name of a first tag found.

This operation is required for SOAP traffic. Name of the first tag after the SOAP:Body tag is often reported as Operation Name.

Configuration:

Tag name 
Attribute name 
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportName

Observed traffic:

Search result:

 XML Attribute example that returns the value of a specific attribute of a first observed tag.
Returns the value of a specific attribute of a first observed tag.

Configuration:

Tag name 
Attribute nameattrname
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportValue

Observed traffic:

Search result:

 XML Attribute example that returns a name of a specific attribute of a first observed tag.
Returns a name of a specific attribute of a first observed tag.

Configuration:

Tag name 
Attribute nameattrname
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportName

Observed traffic:

Search result:

 

XML body search

Use this option to search the body of a SOAP XML structure.

Tag nameThe XML tag name within which you want to search.
Search limitThe number of bytes to search within a file (to limit possibly large searches). The default should suffice in most cases.
Occurrence

Which occurrence of this match you want to return. For example, if you want to find the third occurrence of this match within a file, set Occurrence to 3.

  • Select Any to match any of the occurrences of the defined search step. For example, the given extraction step set as Any will attempt to match the definition of the next step. Only until all subsequent steps are matched the search step set to Any will be considered matched. See an example.
  • Select Count sibling occurrences only to count only the occurrences that happen at the same level within the XML structure, and to ignore any occurrences at other levels within the XML structure.

  • Clear Count sibling occurrences only to count all occurrences regardless of their place in the XML structure.

ReportWhether to return the Name, Value or Any. The Any option will report either the name or the value (whichever is present), and it is the best option to use if you are not certain what the observed XML traffic consists and you wish to avoid blank reports.

Leaving the Tag name empty and setting the Report option, you can search for a specific content and indicate where this content should be found.

 XML Body Search example that returns the value of specified tag.
Returns the value of specified tag.

Given the specific tag name and if the observed traffic is a standard XML structure, the result is the value of the tag. Otherwise, the result is an empty string.

Configuration:

Tag name tagname
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportText

Observed traffic (standard XML structure):

Search result (standard XML structure):

Observed traffic (non standard XML structure):

Search result (non standard XML structure):

(empty string)
 XML Body Search example that returns the content of specified tag if it is a valid child.
Returns the content of specified tag if it is a valid child.

Given the specific tag name and if the observed traffic contains child tags, the result is content of the tag. Otherwise, the result is an empty string.

Configuration:

Tag name tagname
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportBranch

Observed traffic (specified tag does not have child tags):

Search result (specified tag does not have child tags):

(empty string)

Observed traffic (specified tag has child tags):

Search result (specified tag has child tags):

 XML Body Search example that returns the value of the first observed tag of a standard XML format.
Returns the value of the first observed tag of a standard XML format.

Omitted the tag name and if the observed traffic is a standard XML structure, the result is the value of the first observed tag. Otherwise, the result is an empty string.

Configuration:

Tag name 
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportText

Observed traffic (standard XML structure):

Search result (standard XML structure):

Observed traffic (non standard XML structure):

Search result (non standard XML structure):

(empty string)
 XML Body Search example that returns the content of the first observed tag has child tags.
Returns the content of the first observed tag has child tags.

Omitted the tag name and if the observed first observed tag has child tags, the result is the content of the tag. Otherwise, the result is an empty string.

Configuration:

Tag name 
Search limit4096
Occurrence1
Count sibling occurrences onlychecked
ReportBranch

Observed traffic (the first observed tag dos not have child tags):

Search result (standard XML structure):

(empty string)

Observed traffic (non standard XML structure):

Search result (non standard XML structure):


XML sibling element search

Use this option to search for a sibling element in SOAP XML.

Search limitThe number of bytes to search within a file (to limit possibly large searches). The default should suffice in most cases.
Occurrence

Which occurrence of this match you want to return. For example, if you want to find the third occurrence of this match within a file, set Occurrence to 3.

  • Select Any to match any of the occurrences of the defined search step. For example, the given extraction step set as Any will attempt to match the definition of the next step. Only until all subsequent steps are matched the search step set to Any will be considered matched. See an example.
  • Select Count sibling occurrences only to count only the occurrences that happen at the same level within the XML structure, and to ignore any occurrences at other levels within the XML structure.

  • Clear Count sibling occurrences only to count all occurrences regardless of their place in the XML structure.

 

  • No labels
  1. Anonymous (login to see details)

    The 12.3 documentation states that NTLM is supported but not Kerberos. Does the NTLM search now support Kerberos authentication?

    Per the 12.3 docs:

    Note that AMD supports NTLM NTCR - NT Challenge/Response authorization. SPNEGO-based kerberos authentication is not supported.

    1. Anonymous (login to see details)

      John, it was a bug in the docs. Nothing changed here between 12.3 and 12.4. Thank you for pointing this out.

  2. Anonymous (login to see details)

    Note when using "Text search" mode:

    • The "search limit" is an offset constraint. It does not limit the length of possible results. Text that occurs after $limit packet bytes will not be found. The limit does include the header size.
    • The search mode is exclusive. Text entered into match start / end will be stripped from the result. Example: For packet content "prefix_foo_bar_baz_suffix" this feature will find "_bar_" when "foo" and "baz" are specified as match start / end respectively.