Grail DQL in Log Events Extraction

Aymeric_M — Thu, 06 Jun 2024 07:18:42 GMT

I'm in a bit of an dead end : since migrating to Grail the "language" used in the log query of Event Extraction is said to be DQL.

One would assume that we could refine the query in Logs & Events advanced mode then copy/paste it in the Log Query field of Event Extraction, like in the good old time of Log Classic.

But it is not as straightforward as before. The form can not be validated without extensive editing of the query.

It seems that the DQL for Event Extraction is a very limited subset of the "real" DQL.

Anyone would have an idea on how to have the following query working in Event Extraction ?

The documentation is a bit lacking on this subject, so far I guessed that "fetch logs" and the "| filter" keywords should be removed. That leave something like this :

matchesValue(log.source, "frontend") AND matchesValue(type, "secu") AND NOT contains(httpmessage.host, "foo.bar") AND toDouble(userriskdata.score) >= 90 AND matchesValue(attackdata.rules, "USER-RISK-AGGRESSIVE")

That is still not valid: ">", "contains" and "toDouble" are not recognized.

Re: Grail DQL in Log Events Extraction

FranciscoGarcia — Tue, 04 Jun 2024 15:08:51 GMT

You can only use a subset of DQL (I don't think you can use toDouble, etc...). Maybe you can use the new Davis Anomaly detector to create a query that queries logs, makes a timeseries and triggers a David event when the vaue exceeds a static or dinamic threshold?

Re: Grail DQL in Log Events Extraction

Aymeric_M — Wed, 05 Jun 2024 15:01:08 GMT

Thanks for the link to the matcher documentation. I had forgotten about it. I'll look into Davis Anomaly Detecto. In the mean time I'll try to work around contains and > with matchesPhrase.

topic Re: Grail DQL in Log Events Extraction in DQL

Grail DQL in Log Events Extraction

Re: Grail DQL in Log Events Extraction

Re: Grail DQL in Log Events Extraction