topic OpenPipeline summarization in DQL

OpenPipeline summarization

_r_k_ — Thu, 15 May 2025 08:09:32 GMT

How to summarize/aggregate in OpenPipeline

this

```

| summarize accessed_objects_count=sum(arr_size)

```

nor

```

| expand accessed_objects_array
| summarize accessed_objects_count=count()
```

is allowed, so how to summarize/count,

I haven't found any summarization command/function in the documentation:

https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline/reference/openpipeline-dql-functions

https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-processing/lma-log-processing-commands

Can someone provide me with some examples, please?

Re: openpipeline summarization

Fin_Ubels — Thu, 12 Dec 2024 01:28:12 GMT

Hey @_r_k_

What are you trying to achieve with summarization? If you're looking to get a count over time, would turning it into a metric work?

Re: openpipeline summarization

_r_k_ — Thu, 12 Dec 2024 07:34:28 GMT

would you be so kind and provide me with example of self contained openpieline solution containing such aggregation/summarization metric (where attribute can't be directly used, but needs preprocesing first)?

Re: openpipeline summarization

krzysztof_hoja — Thu, 12 Dec 2024 12:08:25 GMT

If you are looking for across multiple log line/event summarization, use of metrics suggestion is correct one. Here is step-by-step example of metric extraction in OpenPipeline: https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline/use-cases/tutorial-log-processing-pipeline

If you want to count number of elements in arrays associated with different ingested objects, first you need to create filed holding array size for single record:

| fieldsAdd accessed_objects_count=arraySize(accessed_objects_array)

and then use this field as input to metric calculation

Re: openpipeline summarization

_r_k_ — Thu, 12 Dec 2024 18:15:43 GMT

yeah, ended up doing exactly this, so...

so we are preparing data in openpipeline and summarization is done outside, no problem, but took a while to realize it