plot linechart from log metric with countDistinctExact

_r_k_ — Thu, 18 Jul 2024 08:21:03 GMT

I cannot figure out how to introduce custom aggregation to log metrics for dashboarding with dynamic timeframe (plot linechart) use

for example countDistinctExact

timeseries log_custom_metric_value = count(log.custom.metric), by: {dimension}

| summarize {no_of_uni=countDistinctExact(log_custom_metric_value[])}, by:{interval, timeframe}

is returning:

This aggregation function currently doesn't support iterative expressions.

is there way how to introduce custom aggregation (for example with summarization) for metrics ?

thank you

Re: plot linechart from log metric with countDistinctExact

krzysztof_hoja — Fri, 19 Jul 2024 11:25:43 GMT

It's true that some aggregation functions are not supported in makeTimeseries command or they do not work for arrays (iterative expressions) in summarize but you can always breakdown array into single rows and do what it needed using summarize and all aggregation functions on any fields and expressions.

Here is example on one of my metrics from log:

timeseries {cnt=count(log.aws.vpc.bytes), timestamp=start()}, by:{vpc_id}, interval:1m | fieldsAdd d=record(cnt=cnt[], timestamp=timestamp[]) | expand d | summarize {cnt=countDistinctExact(d[cnt])}, by: {timestamp=d[timestamp]} | makeTimeseries cnt=max(cnt), interval:1m

First step is to build array of pairs: time and metric value. The next one is to expand it. At this moment each datapoint is individual row and you can apply any aggregation. To get it back as array using makeTimeseries is needed, but as before this step you have each value already in its final form, you can use any aggregation function as this will be 1 filed in on record to 1 element in array conversion (I used max).

I used your query as input where you attempt to calculate distinct count of metric values for each interval.

I hope it helps

Kris

topic plot linechart from log metric with countDistinctExact in DQL

plot linechart from log metric with countDistinctExact

Re: plot linechart from log metric with countDistinctExact