topic convert DQL log query to timeseries in DQL

convert DQL log query to timeseries

roberto_camp1 — Wed, 10 Apr 2024 12:11:48 GMT

So far I have the following DQL query that works:

```

fetch logs
| filter dt.entity.kubernetes_cluster == "KUBERNETES_CLUSTER-ABCDEFG" or in(dt.entity.kubernetes_cluster, "KUBERNETES_CLUSTER-ABCDEFG")
| summarize total = count(), errTotal = countIf(status == "ERROR")

| fieldsAdd errPercent = (toDouble(errTotal)*100 / total)

```

Is it possible to convert this to a timeseries using makeTimeSeries, such that this can be graphed over time or converted to a metric that we can alert on? So far I can't quite get the syntax right for `makeTimeSeries`.

Re: convert DQL log query to timeseries

MichalOlszewski — Wed, 10 Apr 2024 13:50:29 GMT

Hi @roberto_camp1

To create a timeseries from your query you cannot use the "summarize" command, because then you create a summary from all the data that the query returned in the form of a single record.

It is possible to create a timeseries that, for example, draws the number of logs with ERROR status versus the number of all logs, in your case:

fetch logs

| filter dt.entity.kubernetes_cluster == "KUBERNETES_CLUSTER-ABCDEFG" or in(dt.entity.kubernetes_cluster, "KUBERNETES_CLUSTER-ABCDEFG")

| makeTimeseries error_logs = countIf(loglevel == "ERROR"), all_logs = count()

Bests
Michal

Re: convert DQL log query to timeseries

roberto_camp1 — Wed, 10 Apr 2024 15:23:49 GMT

That works for coverting my basic query into timeSeries, so thank you, but my requirement remains to observe the % of error log messsages over time. I have tried to add that to your solution but getting errrors:

,,,

fetch logs

| filter dt.entity.kubernetes_cluster == "KUBERNETES_CLUSTER-763AC7E6FD9C1ED1" or in(dt.entity.kubernetes_cluster, "KUBERNETES_CLUSTER-763AC7E6FD9C1ED1")

| makeTimeseries error_logs = countIf(loglevel == "ERROR"), all_logs = count(), errPercent = (toDouble(error_logs)*100 / all_logs)

'''

ERR MSG: "Mandatory parameter has to be.The parameter has to be an expression-based timeseries aggregation."

Re: convert DQL log query to timeseries

krzysztof_hoja — Wed, 10 Apr 2024 18:30:36 GMT

You need to calculate percentage as a additional step:

| fieldsAdd errPercent = 100*toDouble(error_logs[]) / all_logs[]

Kris