How to trend off log field

[vm_molson]

I'm new to DT, so forgive me if this is too simple. What I am trying to do is create a dashboard which shows "operations" across a time period and only shows ones that:

1. Have degraded over the time period I am looking at
2. Have an average response time greater than some value I set (will be a variable)
3. Appear more than X times per .. day?  (will be a variable)

The response time is a field in our log entries (totalSecs). I added a dashboard portlet using this DQL:

timeseries avg_duration = avg(log.totalsecs), by:{operation}
| filter(arrayAvg(avg_duration) > 2)
| filter(arrayAvg(avg_duration) < 3000)

I'm not sure how to do the first or last piece of the puzzle. I also *think* the filters I have defined are working, but not sure if this syntax is right.

[t_pawlak]

Hi,
In my opinion, your current timeseries query correctly filters by average duration, but it can’t detect degradation over time or enforce “X times per day” conditions.

To do that, you need to:

aggregate per day

compare first vs last daily averages for degradation

apply the count per day threshold

This requires a log-based summarize approach, not just a single timeseries.

Try this:

fetch logs
| filter isNotNull(totalSecs)

| fieldsAdd
    duration = toDouble(totalSecs),
    day = formatTimestamp(timestamp, format:"yyyy-MM-dd")

| summarize
    avg_duration = avg(duration),
    first_seen   = min(duration),
    last_seen    = max(duration),
    per_day      = count(),
    by:{operation, day}

| summarize
    avg_duration = avg(avg_duration),
    total_count  = sum(per_day),
    first_seen   = min(first_seen),
    last_seen    = max(last_seen),
    by:{operation}

| fieldsAdd
    degraded = last_seen > first_seen

| filter degraded == true
| filter avg_duration > 2
| filter total_count > 10

This requires a log-based summarize approach, not just a single timeseries.