Create a log processing rule to keep the latest log received from the timestamp

[AlexisGND]

Hello,
I want to make a processing rule to filter out an entire log.

Here's the log sample :

[
    {
      "interface.name": "Seq_SAP_to_AGV",
      "host.environment": "PROD",
      "host.name": "Datastage",
      "content": "Seq_SAP_to_AGV Finished - OK",
      "execution.status": "Finished - OK",
      "execution.starttime": "2024-05-28 14:00:34",
      "execution.endtime": "2024-05-28 14:01:34",
      "event.type": "LOG",
      "loglevel": "NOTICE",
      "interface.target.domain": "MFG",
      "ID": "Seq_SAP_to_AGV#28052024.140034",
      "timestamp": "2024-05-28T14:00:34.000000000Z",
      "status": "INFO"
    },
    {
      "interface.name": "Seq_SAP_to_AGV",
      "host.environment": "PROD",
      "host.name": "Datastage",
      "content": "Seq_SAP_to_AGV Running - no warnings",
      "execution.status": "Running - no warnings",
      "execution.starttime": "2024-05-28 14:00:34",
      "event.type": "LOG",
      "loglevel": "NOTICE",
      "interface.target.domain": "MFG",
      "ID": "Seq_SAP_to_AGV#28052024.140034",
      "timestamp": "2024-05-28T14:00:34.000000000Z",
      "status": "INFO"
    }
  ]

As you can see, the "ID" is the same between these 2 logs because i received a "Running" status first and a "Finished" status at the end.

Now my need is to filter out all of the other informations who contains "Running with..." with this ID because it is an outdated information since I received the same one with the "Finished with..." and the "endtime".

If I want to speak with other words : I want to keep only the latest information received from the timestamp for every ID.

Is it possible ?
Thanks in advance 🙂

Alexis

[Malaik]

Hi,

I have a similar need (But mine is a little complex).

For your case I can suggest to drop logs that contain "Running" ... and with this rule you will keep only "Finished with..." and the "endtime"...

Hope this help.

[AlexisGND]

Hello,

Thanks for your suggestion but unfortunately my need is to also display the running datas for those who are actually running in my dashboards 😕

But the idea of making multiple queries, then to be able to assemble them in one DataTable is possible with my NodeJs custom app on Dynatrace.

I will try this method instead until i find something relevant to this log processing rule.

Thanks again 😉
Alexis