I have a query into my logs that looks something like this:
fetch log
| filter matchesValue(dt.entity.process_group_instance, "the-process-im-looking-at")
| parse content, "TIMESTAMP:DATETIME LD ' 443 - ' LD:REQADDR ' ' LD ' - 200 ' LD ' ' LD ' ' LD:DURATION"
| fields timestamp, DATETIME, REQADDR, DURATION=toLong(DURATION), content
The log lines describe a request to an endpoint; each contains a datetime at the beginning, some other info in the middle, and a duration of the request in milliseconds at the end. The application cannot be deep-monitored, due to the technology choices made (enabling deep monitoring brings the whole application down), so we can't get this info the normal way; we can only harvest from the logs. So, if a request took 5 minutes, it'd look something like this:
2025-12-23 17:52:23 10.0.0.23 POST /application/api.json abcd 443 - 12.21.12.21 client-identifier - 200 0 0 300000
There's a need to know how many requests are open at the same time over some time window--let's describe a scenario to explain what we mean:
1. No requests are open at 16:29. A request is opened at 16:30; we'll call it `A`. This request takes 10 minutes to complete.
2. Two requests are opened at 16:35. These requests take 10 minutes to complete. These are called `B` and `C`.
3. `A` closes. Five new requests open between 16:40 and 16:45. These requests all take 10 milliseconds each to complete, and only 3 overlap; the other two open sequentially afterwards. These are `D`, `E`, `F`, `G`, `H`.
4. We make a timeseries with 5-minute buckets, based on info we get from parsing the logs; we want to see "1" for 16:30-16:35, "3" for 16:35-16:40, "5" or "7" for 16:40-16:45, and then "0" for 16:45 onwards.
Ideally, we would see "5" in the 16:40-16:45 bucket; this is because only 5 at most overlapped at any one time (B, C, D, E, F); however, it's also acceptable to do "7", since 7 total were active in that bucket. Is there a way to do this in DQL currently? Is there a simpler option I am missing that works in a much simpler fashion (without deep monitoring)?
I could not get this working in my experiments--I ended up pulling the parsed data down as CSV, and using Python to find the overlaps over a timeseries. This worked accurately, but it would be ideal to do this all in Dynatrace; it would enable things like graphs, alerts, and so on, which would be useful for our case.
