This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
DQL
Questions about Dynatrace Query Language
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

PARSING INTO JSON_ARRAY - Different behavior between openpipeline and notebook

FredSelvaGen
Observer

‎08 Jun 2026 11:13 AM

Hello community,

I'm looking for help regarding bizevent processing in openpipeline.

It looks like that PARSE (into JSON_ARRAY)  dql command doesn't have the same behavior between Openpipeline and Notebook

 

Data input :  

  • We ingest Cart contents as a bizevent 
  • The cart content format is a JSON_ARRAY that been ingested as a String
  • The array could have 1 or N elements.

Cart content Raw (in string) : 

"input_array_in_string": "[{\"cartid\":\"202605270404285529\",\"itemType\":\"ITEM1\",\"itemAmount\":100},{\"cartid\":\"20260527040428553\",\"itemType\":\"ITEM2\",\"itemAmount\":200},{\"cartid\":\"20260527040456789\",\"itemType\":\"ITEM1\",\"itemAmount\":300},{\"cartid\":\"202605270404987654\",\"itemType\":\"ITEM3\",\"itemAmount\":400},{\"cartid\":\"2026052704654789\",\"itemType\":\"ITEM2\",\"itemAmount\":500},{\"cartid\":\"202605271098431\",\"itemType\":\"ITEM1\",\"itemAmount\":600}]"

 

Cart content (Structured as an array) : 

      [
        {
          "cartid": "202605270404285529",
          "itemType": "ITEM1",
          "itemAmount": "100"
        },
        {
          "cartid": "20260527040428553",
          "itemType": "ITEM2",
          "itemAmount": "200"
        },
        {
          "cartid": "20260527040456789",
          "itemType": "ITEM1",
          "itemAmount": "300"
        },
        {
          "cartid": "202605270404987654",
          "itemType": "ITEM3",
          "itemAmount": "400"
        },
        {
          "cartid": "2026052704654789",
          "itemType": "ITEM2",
          "itemAmount": "500"
        },
        {
          "cartid": "202605271098431",
          "itemType": "ITEM1",
          "itemAmount": "600"
        }
      ]

 

My need : 

  • I would like to get into new fields :
    • the number of cart related to "ITEM1" (= 3)
    • the amount value for "ITEM1" (= 1000)

 

Found solution (tested OK in a notebook) :

Because of openpipeline limitation, I found a way to get the expected informations (Item1 cart amount and cart count) without using EXPAND dql command 

// PARSE STRING AS JSON_ARRAY
| parse input_array_in_string, """JSON_ARRAY:output_array_in_notebook"""


// COLLECT IN A NEW ARRAY EACH CART ELEMENT CONTAINING ITEM1
| fieldsAdd notebook_array_of_item1 = arrayRemoveNulls( iCollectArray(if(output_array_in_notebook[][itemType]=="ITEM1",output_array_in_notebook[])))

// CACULATE VALUES : COUNT AND AMOUNT OF ITEM1 CARTS
| fieldsAdd notebook_item1_arraySize = arraySize(notebook_array_of_item1)
| fieldsAdd notebook_item1_cumulativeAmount =arraySum(iCollectArray(notebook_array_of_item1[][itemAmount]))

 

Result in a notebook

{
  "records": [
    {
      "timestamp": "2026-06-08T10:33:48.836000000+02:00",
      "output_array_in_notebook": [
        {
          "cartid": "202605270404285529",
          "itemType": "ITEM1",
          "itemAmount": "100"
        },
        {
          "cartid": "20260527040428553",
          "itemType": "ITEM2",
          "itemAmount": "200"
        },
        {
          "cartid": "20260527040456789",
          "itemType": "ITEM1",
          "itemAmount": "300"
        },
        {
          "cartid": "202605270404987654",
          "itemType": "ITEM3",
          "itemAmount": "400"
        },
        {
          "cartid": "2026052704654789",
          "itemType": "ITEM2",
          "itemAmount": "500"
        },
        {
          "cartid": "202605271098431",
          "itemType": "ITEM1",
          "itemAmount": "600"
        }
      ],
      "notebook_array_of_item1": [
        {
          "cartid": "202605270404285529",
          "itemType": "ITEM1",
          "itemAmount": "100"
        },
        {
          "cartid": "20260527040456789",
          "itemType": "ITEM1",
          "itemAmount": "300"
        },
        {
          "cartid": "202605271098431",
          "itemType": "ITEM1",
          "itemAmount": "600"
        }
      ],
      "notebook_item1_arraySize": "3",
      "notebook_item1_cumulativeAmount": 1000,

     }
  ],
  "types": [
    {
      "indexRange": [
        0,
        1
      ],
      "mappings": {
        "timestamp": {
          "type": "timestamp"
        },
        "output_array_in_notebook": {
          "type": "array",
          "types": [
            {
              "indexRange": [
                0,
                5
              ],
              "mappings": {
                "element": {
                  "type": "record",
                  "types": [
                    {
                      "mappings": {
                        "cartid": {
                          "type": "string"
                        },
                        "itemType": {
                          "type": "string"
                        },
                        "itemAmount": {
                          "type": "long"
                        }
                      }
                    }
                  ]
                }
              }
            }
          ]
        },
        "notebook_array_of_item1": {
          "type": "array",
          "types": [
            {
              "indexRange": [
                0,
                2
              ],
              "mappings": {
                "element": {
                  "type": "record",
                  "types": [
                    {
                      "mappings": {
                        "cartid": {
                          "type": "string"
                        },
                        "itemType": {
                          "type": "string"
                        },
                        "itemAmount": {
                          "type": "long"
                        }
                      }
                    }
                  ]
                }
              }
            }
          ]
        },
        "notebook_item1_arraySize": {
          "type": "long"
        },
        "notebook_item1_cumulativeAmount": {
          "type": "double"
        },
        "input_array_in_string": {
          "type": "string"
        }
      }
    }
  ]
}

 

Unfortunately, I don't get any result when using the same commands in a openpipeline processor  

 

DQL Process definition

parse input_array_in_string, """JSON_ARRAY:output_array"""

| fieldsAdd openpipeline_array_of_item1 = arrayRemoveNulls( iCollectArray(if(output_array[][itemType]=="ITEM1",output_array[])))

| fieldsAdd openpipeline_item1_arraySize = arraySize(openpipeline_array_of_item1)
| fieldsAdd openpipeline_item1_cumulativeAmount =arraySum(iCollectArray(openpipeline_array_of_item1[][itemAmount]))

 

Openpipeline result 

{
  "records": [
    {
      "timestamp": "2026-06-08T10:33:48.836000000+02:00",
      "input_array_in_string": "[{\"cartid\":\"202605270404285529\",\"itemType\":\"ITEM1\",\"itemAmount\":100},{\"cartid\":\"20260527040428553\",\"itemType\":\"ITEM2\",\"itemAmount\":200},{\"cartid\":\"20260527040456789\",\"itemType\":\"ITEM1\",\"itemAmount\":300},{\"cartid\":\"202605270404987654\",\"itemType\":\"ITEM3\",\"itemAmount\":400},{\"cartid\":\"2026052704654789\",\"itemType\":\"ITEM2\",\"itemAmount\":500},{\"cartid\":\"202605271098431\",\"itemType\":\"ITEM1\",\"itemAmount\":600}]",
      "id": "2",
      "event.id": "d44d5a05-c6d5-4881-b04f-73e3caf5b88f",
      "event.kind": "BIZ_EVENT",
      "dt.openpipeline.source": "/api/v2/bizevents/ingest",
      "dt.openpipeline.pipelines": [
        "bizevents:pipeline_parse_json_array_5188"
      ],
      "output_array": [
        "{\"cartid\":\"202605270404285529\", \"itemType\":\"ITEM1\", \"itemAmount\":100}",
        "{\"cartid\":\"20260527040428553\", \"itemType\":\"ITEM2\", \"itemAmount\":200}",
        "{\"cartid\":\"20260527040456789\", \"itemType\":\"ITEM1\", \"itemAmount\":300}",
        "{\"cartid\":\"202605270404987654\", \"itemType\":\"ITEM3\", \"itemAmount\":400}",
        "{\"cartid\":\"2026052704654789\", \"itemType\":\"ITEM2\", \"itemAmount\":500}",
        "{\"cartid\":\"202605271098431\", \"itemType\":\"ITEM1\", \"itemAmount\":600}"
      ],
      "openpipeline_array_of_item1": [],
      "openpipeline_item1_arraySize": "0",
      "openpipeline_item1_cumulativeAmount": null
    }
  ],
  "types": [
    {
      "indexRange": [
        0,
        1
      ],
      "mappings": {
        "timestamp": {
          "type": "timestamp"
        },
        "input_array_in_string": {
          "type": "string"
        },
        "id": {
          "type": "string"
        },
        "event.id": {
          "type": "string"
        },
        "event.kind": {
          "type": "string"
        },
        "dt.openpipeline.source": {
          "type": "string"
        },
        "dt.openpipeline.pipelines": {
          "type": "array",
          "types": [
            {
              "indexRange": [
                0,
                0
              ],
              "mappings": {
                "element": {
                  "type": "string"
                }
              }
            }
          ]
        },
        "output_array": {
          "type": "array",
          "types": [
            {
              "indexRange": [
                0,
                5
              ],
              "mappings": {
                "element": {
                  "type": "string"
                }
              }
            }
          ]
        },
        "openpipeline_array_of_item1": {
          "type": "array",
          "types": []
        },
        "openpipeline_item1_arraySize": {
          "type": "long"
        },
        "openpipeline_item1_cumulativeAmount": {
          "type": "double"
        }
      }
    }
  ]
}

 

 

Possible root cause

It seems that PARSE DQL command doesn't give the same result between Notebook and openpipeline

 

  1. Do you have encounter the same issue with openpipeline (product bug?)
  2. Do you have any idea for solving this issue?

 

Difference between notebook and openpipeline

//NOTEBOOK
      "output_array_in_notebook": [
        {
          "cartid": "202605270404285529",
          "itemType": "ITEM1",
          "itemAmount": "100"
        },
        {
          "cartid": "20260527040428553",
          "itemType": "ITEM2",
          "itemAmount": "200"
        },
        {
          "cartid": "20260527040456789",
          "itemType": "ITEM1",
          "itemAmount": "300"
        },
        {
          "cartid": "202605270404987654",
          "itemType": "ITEM3",
          "itemAmount": "400"
        },
        {
          "cartid": "2026052704654789",
          "itemType": "ITEM2",
          "itemAmount": "500"
        },
        {
          "cartid": "202605271098431",
          "itemType": "ITEM1",
          "itemAmount": "600"
        }
      ],



// OPENPIPELINE
      "output_array": [
        "{\"cartid\":\"202605270404285529\", \"itemType\":\"ITEM1\", \"itemAmount\":100}",
        "{\"cartid\":\"20260527040428553\", \"itemType\":\"ITEM2\", \"itemAmount\":200}",
        "{\"cartid\":\"20260527040456789\", \"itemType\":\"ITEM1\", \"itemAmount\":300}",
        "{\"cartid\":\"202605270404987654\", \"itemType\":\"ITEM3\", \"itemAmount\":400}",
        "{\"cartid\":\"2026052704654789\", \"itemType\":\"ITEM2\", \"itemAmount\":500}",
        "{\"cartid\":\"202605271098431\", \"itemType\":\"ITEM1\", \"itemAmount\":600}"
      ],

 

 

Thank you

Freddy

Labels:
0 Kudos
Reply
0 REPLIES 0
Ask a question

Featured Posts