Latency calculation based on logs timestamp

[VladS]

Hello!

I have some logs with this general structure:

{timestamp: t1, logpoint:"request-out", transaction-id: tx1, other stuff}

{timestamp: t2, logpoint:"response-in", transaction-id: tx1, other stuff}

{timestamp: t3, logpoint:"request-out", transaction-id: tx2, other stuff}

{timestamp: t4, logpoint:"response-in", transaction-id: tx2, other stuff}

I need to plot a chart with the latency (duration) of each transaction-id, so essentially the difference between timestamps t2 and t1 for tx1, then t4 - t3 for tx2, and so on. Therefore I would need a query that returns a timeseries containing the values for the latency on the y axis, and either transaction-ids or just timestamp for response-in on the y axis.

Any suggestion is appreciated. Thanks!

[marco_irmer · Accepted solution]

This can be accomplished by first sumamrizing on the transaction id, then calculating the duration, and finally using the makeTimeSeries command to plot this over time.

Here's some sample DQL to get you started

// dummy data for testing purposes
data record(transaction_id="tx1", logpoint="response-in", timestamp = now()-20m),
     record(transaction_id="tx1", logpoint="request-out", timestamp = now()-21m),
     record(transaction_id="tx2", logpoint="response-in", timestamp = now()-10m),
     record(transaction_id="tx2", logpoint="request-out", timestamp = now()-12m)

// summarize to create a single record containing the start and end time of each transaction
| summarize { timestamp.start = min(timestamp), 
              timestamp.end = max(timestamp)
            },
            by: { transaction_id }

// calculate duration
| fieldsAdd duration = timestamp.end - timestamp.start

// create a timeseries
| makeTimeseries {interval.start = start(),
                  interval.end = end(),
                  duration = avg(duration)
                  },
                  time:timestamp.start,
                  nonempty:true,
                  interval: 1m,
                  by:{ transaction_id }

[VladS]

Thank you very much! Exactly what I needed