openpipeline summarization

[_r_k_]

how to summarize / aggregate in openpipeline

this

```

| summarize accessed_objects_count=sum(arr_size)

```

nor

```

| expand accessed_objects_array
| summarize accessed_objects_count=count()
```

is allowed, so how to summarize / count 

i haven't found any summarization command/function in documentation:

https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline/reference/openpipeline-dql-...

https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-processing/lma-log-processing-...

can someone provide me with some example please ?

[Fin_Ubels]

Hey @_r_k_ 

What are you trying to achieve with summarization? If you're looking to get a count over time, would turning it into a metric work? 

[_r_k_]

would you be so kind and provide me with example of self contained openpieline solution containing such aggregation/summarization metric (where attribute can't be directly used, but needs preprocesing first)?

[krzysztof_hoja · Accepted solution]

If you are looking for across multiple log line/event summarization, use of metrics suggestion is correct one. Here is step-by-step example of metric extraction in OpenPipeline: https://docs.dynatrace.com/docs/discover-dynatrace/platform/openpipeline/use-cases/tutorial-log-proc...

If you want to count number of elements in arrays associated with different ingested objects, first you need to create filed holding array size for single record:

| fieldsAdd accessed_objects_count=arraySize(accessed_objects_array)

and then use this field as input to metric calculation

[_r_k_]

yeah, ended up doing exactly this, so...

so we are preparing data in openpipeline and summarization is done outside, no problem, but took a while to realize it