topic Re: Latency calculation based on logs timestamp in DQL

Latency calculation based on logs timestamp

VladS — Tue, 01 Apr 2025 11:19:30 GMT

Hello!

I have some logs with this general structure:

{timestamp: t1, logpoint:"request-out", transaction-id: tx1, other stuff}

{timestamp: t2, logpoint:"response-in", transaction-id: tx1, other stuff}

{timestamp: t3, logpoint:"request-out", transaction-id: tx2, other stuff}

{timestamp: t4, logpoint:"response-in", transaction-id: tx2, other stuff}

I need to plot a chart with the latency (duration) of each transaction-id, so essentially the difference between timestamps t2 and t1 for tx1, then t4 - t3 for tx2, and so on. Therefore I would need a query that returns a timeseries containing the values for the latency on the y axis, and either transaction-ids or just timestamp for response-in on the y axis.

Any suggestion is appreciated. Thanks!

Re: Latency calculation based on logs timestamp

marco_irmer — Tue, 01 Apr 2025 23:06:24 GMT

This can be accomplished by first sumamrizing on the transaction id, then calculating the duration, and finally using the makeTimeSeries command to plot this over time.

Here's some sample DQL to get you started

// dummy data for testing purposes data record(transaction_id="tx1", logpoint="response-in", timestamp = now()-20m), record(transaction_id="tx1", logpoint="request-out", timestamp = now()-21m), record(transaction_id="tx2", logpoint="response-in", timestamp = now()-10m), record(transaction_id="tx2", logpoint="request-out", timestamp = now()-12m) // summarize to create a single record containing the start and end time of each transaction | summarize { timestamp.start = min(timestamp), timestamp.end = max(timestamp) }, by: { transaction_id } // calculate duration | fieldsAdd duration = timestamp.end - timestamp.start // create a timeseries | makeTimeseries {interval.start = start(), interval.end = end(), duration = avg(duration) }, time:timestamp.start, nonempty:true, interval: 1m, by:{ transaction_id }

Re: Latency calculation based on logs timestamp

VladS — Wed, 02 Apr 2025 05:35:30 GMT

Thank you very much! Exactly what I needed