Extract multiple values from XML logs where field names may vary

Kajao — Mon, 17 Nov 2025 09:39:30 GMT

I have XML-based logs from an API integration that look like this (simplified example):

2025-11-03 09:00:01,445 lLv="INFO" [<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Header> <ns1:SourceApplication>myApp</ns1:SourceApplication> <ns1:Function>Payment</ns1:Function> </soapenv:Header> <soapenv:Body> <PaymentInitiateApproveUpdateRequest> <PaymentInitiateApproveRequest> <ns8:TransactionNumber xmlns:ns8="x:x">111111</ns8:TransactionNumber> <ns9:SerialNumber xmlns:ns9="x:x">22222222</ns9:SerialNumber> </PaymentInitiateApproveRequest> <PaymentInitiateApproveRequest> <ns19:TransactionNumber xmlns:ns19="x:x">333333</ns19:TransactionNumber> <ns20:SerialNumber xmlns:ns20="x:x">44444444</ns20:SerialNumber> </PaymentInitiateApproveRequest> </PaymentInitiateApproveUpdateRequest> </soapenv:Body> </soapenv:Envelope>], response: [<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <ns4:PaymentInitiateApproveUpdateResponse <ns5:ResponseState> <ErrorCode>0</ErrorCode> </ns5:ResponseState> <ns4:PaymentInitiateApproveResponse> <ns2:PaymentInitiateKey> <ns3:TransactionNumber>555555</ns3:TransactionNumber> <ns3:SerialNumber>66666666</ns3:SerialNumber> </ns2:PaymentInitiateKey> <ns2:ServiceElementResult> <ns2:ErrorCode>0</ns2:ErrorCode> </ns2:ServiceElementResult> </ns4:PaymentInitiateApproveResponse> <ns4:PaymentInitiateApproveResponse> <ns2:PaymentInitiateKey> <ns3:TransactionNumber>777777</ns3:TransactionNumber> <ns3:SerialNumber>888888888</ns3:SerialNumber> </ns2:PaymentInitiateKey> <ns2:ServiceElementResult> <ns2:ErrorCode>0</ns2:ErrorCode> </ns2:ServiceElementResult> </ns4:PaymentInitiateApproveResponse> </ns4:PaymentInitiateApproveUpdateResponse> </soap:Body> </soap:Envelope>]."

I need to extract fields like:

Function from the header.
TransactionNumber and SerialNumber from the request.
ErrorCode from the response (there can be multiple for payments and one for transfer)

Challenges:

These elements can appear multiple times (arrays of PaymentInitiateApproveRequest and PaymentInitiateApproveResponse).
The XML tags vary, for example SerialNumber might appear as:
- <ns9:SerialNumber>614276066</ns9:SerialNumber>
- <ns1:SerialNumber>614276066</ns1:SerialNumber>
- <SerialNumber>614276066</SerialNumber>
My main problem is extracting the ErrorCode from the payment. It always appears after ns2:ServiceElementResult:

<ns2:ServiceElementResult> <ns2:ErrorCode>0</ns2:ErrorCode>

This approach work for all the fields except the payment ErrorCode:

| parse content, """Array{DATA 'TransactionNumbe' LD '>' LONG*:i '</' LD 'ransactionNumber>'LF}{1,}:trans_num """ | parse content, """Array{DATA 'SerialNumbe' LD '>' LONG*:i '</' LD 'erialNumber>'LF}{1,}:ser_num """ | parse content, """DATA "ErrorCode>" DATA:error_code_transfer "</ErrorCode>" """ | parse content, """Array{DATA 'ServiceElementResult>' EOL LD 'ErrorCode>' LONG*:i '</' LD 'ErrorCode>' LF}{1,}:error_code_payment """ // does not work

I have also tried with the XML approach:

| parse content, """ DATA '[' XML_PLAIN:xml_env ']' DATA '[' XML_PLAIN:xml_response """ | fieldsAdd env_header = xml_env[`soapenv:Envelope`][`soapenv:Header`], reponse_body = xml_response[`soap:Envelope`][`soap:Body`]

But I am not able to handle varying tag names (namespace prefixes) and multiple values with this approach.

Re: Extract multiple values from XML logs where field names may vary

Tiit_Hallas — Tue, 18 Nov 2025 22:39:53 GMT

I would use the following approach (ignoring namespaces and dealing with multiple values):

| parse content, "LD XML_PLAIN(ignoreNamespace=true):request ld '[' XML(ignoreNamespace=true):response"

Additionally you can use the expand and fieldsFlatten commands to open up the records and arrays to separate result records. A solution can be seen in Security Investigator on our Playground at https://wkf10640.apps.dynatrace.com/ui/apps/dynatrace.security.investigator/share/9bff51ab-29fd-472e-a96d-49ce58453d60

topic Extract multiple values from XML logs where field names may vary in DQL

Extract multiple values from XML logs where field names may vary

Re: Extract multiple values from XML logs where field names may vary