Extract multiple values from XML logs where field names may vary

[Kajao]

I have XML-based logs from an API integration that look like this (simplified example):

2025-11-03 09:00:01,445 lLv="INFO" [<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 <soapenv:Header>
   <ns1:SourceApplication>myApp</ns1:SourceApplication>
   <ns1:Function>Payment</ns1:Function>
 </soapenv:Header>
 <soapenv:Body>
  <PaymentInitiateApproveUpdateRequest>
   <PaymentInitiateApproveRequest>
     <ns8:TransactionNumber xmlns:ns8="x:x">111111</ns8:TransactionNumber>
     <ns9:SerialNumber xmlns:ns9="x:x">22222222</ns9:SerialNumber>
   </PaymentInitiateApproveRequest>
   <PaymentInitiateApproveRequest>
     <ns19:TransactionNumber xmlns:ns19="x:x">333333</ns19:TransactionNumber>
     <ns20:SerialNumber xmlns:ns20="x:x">44444444</ns20:SerialNumber>
   </PaymentInitiateApproveRequest>
  </PaymentInitiateApproveUpdateRequest>
 </soapenv:Body>
</soapenv:Envelope>], response: [<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
 <soap:Body>
  <ns4:PaymentInitiateApproveUpdateResponse 
   <ns5:ResponseState>
    <ErrorCode>0</ErrorCode>
   </ns5:ResponseState>
   <ns4:PaymentInitiateApproveResponse>
    <ns2:PaymentInitiateKey>
     <ns3:TransactionNumber>555555</ns3:TransactionNumber>
     <ns3:SerialNumber>66666666</ns3:SerialNumber>
    </ns2:PaymentInitiateKey>
    <ns2:ServiceElementResult>
     <ns2:ErrorCode>0</ns2:ErrorCode>
    </ns2:ServiceElementResult>
   </ns4:PaymentInitiateApproveResponse>
   <ns4:PaymentInitiateApproveResponse>
    <ns2:PaymentInitiateKey>
     <ns3:TransactionNumber>777777</ns3:TransactionNumber>
     <ns3:SerialNumber>888888888</ns3:SerialNumber>
    </ns2:PaymentInitiateKey>
    <ns2:ServiceElementResult>
     <ns2:ErrorCode>0</ns2:ErrorCode>
    </ns2:ServiceElementResult>
   </ns4:PaymentInitiateApproveResponse>
  </ns4:PaymentInitiateApproveUpdateResponse>
 </soap:Body>
</soap:Envelope>]."

 
I need to extract fields like:

• Function from the header.
• TransactionNumber and SerialNumber from the request.
• ErrorCode from the response (there can be multiple for payments and one for transfer)

Challenges:

• These elements can appear multiple times (arrays of PaymentInitiateApproveRequest and PaymentInitiateApproveResponse).
• The XML tags vary, for example SerialNumber might appear as:
  • <ns9:SerialNumber>614276066</ns9:SerialNumber>
  • <ns1:SerialNumber>614276066</ns1:SerialNumber>
  • <SerialNumber>614276066</SerialNumber>
• My main problem is extracting the ErrorCode from the payment. It always appears after ns2:ServiceElementResult:

<ns2:ServiceElementResult>
 <ns2:ErrorCode>0</ns2:ErrorCode>

This approach work for all the fields except the payment ErrorCode:

| parse content, """Array{DATA 'TransactionNumbe' LD '>' LONG*:i '</' LD 'ransactionNumber>'LF}{1,}:trans_num """
| parse content, """Array{DATA 'SerialNumbe' LD '>' LONG*:i '</' LD 'erialNumber>'LF}{1,}:ser_num """
| parse content, """DATA "ErrorCode>" DATA:error_code_transfer "</ErrorCode>" """
| parse content, """Array{DATA 'ServiceElementResult>' EOL LD 'ErrorCode>' LONG*:i '</' LD 'ErrorCode>' LF}{1,}:error_code_payment """ // does not work

I have also tried with the XML approach:

| parse content, """ DATA '[' XML_PLAIN:xml_env ']' DATA '[' XML_PLAIN:xml_response """
| fieldsAdd env_header = xml_env[`soapenv:Envelope`][`soapenv:Header`], reponse_body = xml_response[`soap:Envelope`][`soap:Body`]

But I am not able to handle varying tag names (namespace prefixes) and multiple values with this approach.

[Tiit_Hallas · Accepted solution]

I would use the following approach (ignoring namespaces and dealing with multiple values):

| parse content, "LD XML_PLAIN(ignoreNamespace=true):request ld '[' XML(ignoreNamespace=true):response"

Additionally you can use the expand and fieldsFlatten commands to open up the records and arrays to separate result records. A solution can be seen in Security Investigator on our Playground at https://wkf10640.apps.dynatrace.com/ui/apps/dynatrace.security.investigator/share/9bff51ab-29fd-472e...