Metric anomaly detection for a range of values in the logs

davidrb — Mon, 26 Aug 2024 05:50:27 GMT

Attempting to create an Anomaly detection metric event based on a specific range of data from logs. I can find this data from the logs with this DQL query:

fetch logs | filter matchesValue(k8s.namespace.name, "app-namespace") and toLong(application-event-id) >= 100 and toLong(application-event-id) <= 1000 | sort timestamp desc

I am creating a Log Metric, that will be used by a Anomaly detection metric event, but the Matcher doesn't allow the range selection: toLong(application-event-id) >= 100 and toLong(application-event-id) <= 1000. I am using this instead:

matchesValue(k8s.namespace.name, "app-namespace") and not isNull("application-event-id")

Then using an Anomaly detection metric event to raise a Problem if a certain threshold of 100-1000 application-event-

id are raised in the logs.

I can filter for a specific application-event-id, but how can I do this for the range of application-event-id in the logs (100 to 1000)

log.application_event_ids:filter(eq("application-eventid-id",100))

Re: Metric anomaly detection for a range of values in the logs

DavidBruendl — Thu, 21 Nov 2024 10:20:56 GMT

Hi @davidrb ,

you can use your log query add |makeTimeseries and place it in the Davis Anomaly Detector

fetch logs | filter matchesValue(k8s.namespace.name, "app-namespace") and toLong(application-event-id) >= 100 and toLong(application-event-id) <= 1000 | makeTimeseries count()

Please give a try, thank you 🙂

best greetings

David

topic Metric anomaly detection for a range of values in the logs in DQL

Metric anomaly detection for a range of values in the logs

Re: Metric anomaly detection for a range of values in the logs