DQL Regex

[ppaulogustavoem]

Hey folks! 

DQL is amazing. It works very well when the logs is in a json format. However, I have a case where the application log is in a complete wild format.

[critical] consumer.go:117 [!dt dt.trace_id=xxxxx,dt.span_id=xxxx][request_id=xxxx][session_id=][user_id=xxxxxxx]Queue: xxxxxxxx~~~Message type: createQuoteSummaryTask~~~~~~Error:~~~Code: Some Error here ~~~Message: ~~~Stack:~~~goroutine 315080

I know the format is a huge problem, however, I would like to extract a very specific piece of this log using Regex. 

Is this possible? I tried to insert a new DPL pattern but I couldn't understand very well if it would be possible just run a simple regex.

What I want to do is run the following regex along with the parser command: `Error:(.*?)Message:`.

Is this possible? 

[krzysztof_hoja · Accepted solution]

You cannot use regex in DPL, but DPL is a pattern matching and parsing language which replaces regex and does the job easier.

Actually such format is not a problem:

data record(content="[critical] consumer.go:117 [!dt dt.trace_id=xxxxx,dt.span_id=xxxx][request_id=xxxx][session_id=][user_id=xxxxxxx]Queue: xxxxxxxx~~~Message type: createQuoteSummaryTask~~~~~~Error:~~~Code: Some Error here ~~~Message: ~~~Stack:~~~goroutine 315080")
| parse content, "LD 'Error:' LD:error_message 'Message:'"

Of course if you want to take out other fields from such a log line it is possible as well as making the pattern more precise, so it does not catch similar log lines

Kris