Grail DQL in Log Events Extraction

[Aymeric_M]

I'm in a bit of an dead end : since migrating to Grail the "language" used in the log query of Event Extraction is said to be DQL.

One would assume that we could refine the query in Logs & Events advanced mode then copy/paste it in the Log Query field of Event Extraction, like in the good old time of Log Classic.

But it is not as straightforward as before. The form can not be validated without extensive editing of the query.

It seems that the DQL for Event Extraction is a very limited subset of the "real" DQL.

Anyone would have an idea on how to have the following query working in Event Extraction ?

fetch logs
| filter matchesValue(log.source, "frontend")
| filter matchesValue(type, "secu")
| filterOut  contains(httpmessage.host, "foo.bar")
| filter toDouble(userriskdata.score) >= 90
| filter matchesValue(attackdata.rules, "USER-RISK-AGGRESSIVE")

The documentation is a bit lacking on this subject, so far I guessed that "fetch logs" and the "| filter" keywords should be removed. That leave something like this :

matchesValue(log.source, "frontend")
AND matchesValue(type, "secu")
AND NOT  contains(httpmessage.host, "foo.bar")
AND toDouble(userriskdata.score) >= 90
AND matchesValue(attackdata.rules, "USER-RISK-AGGRESSIVE")

That is still not valid: ">", "contains" and "toDouble" are not recognized.

[FranciscoGarcia]

You can only use a subset of DQL (I don't think you can use toDouble, etc...). Maybe you can use the new Davis Anomaly detector to create a query that queries logs, makes a timeseries and triggers a David event when the vaue exceeds a static or dinamic threshold?

[Aymeric_M]

Thanks for the link to the matcher documentation. I had forgotten about it. I'll look into Davis Anomaly Detecto. In the mean time I'll try to work around contains and > with matchesPhrase.