topic Re: Is there a reverse command of fieldsFlatten? in DQL

Is there a reverse command of fieldsFlatten?

lucassilveira — Mon, 21 Oct 2024 14:45:55 GMT

I would like to make a json from some log attributes. Is this possible?

Re: Is there a reverse command of fieldsFlatten?

krzysztof_hoja — Mon, 21 Oct 2024 16:11:29 GMT

Yes, if you can enumerate them. Let's assume you have following demo data:

You can build object (complex record) using record function:

Here is my query for easy reuse:

data json:"""[{"a":1,"b":"x"},{"a":2,"b":"y"}]""" | fieldsAdd j = record(a,b) | fieldsAdd js = toString(j)

I hope it helps

Re: Is there a reverse command of fieldsFlatten?

MostafaHussein — Tue, 22 Oct 2024 06:11:06 GMT

Hello @lucassilveira ,

can you mention sample log line to give you the proper transformation method?

Re: Is there a reverse command of fieldsFlatten?

Mohamed_Hamdy — Tue, 22 Oct 2024 06:40:45 GMT

In addition to what @krzysztof_hoja mentioned, you can use Key-Value Pairs (KVPs) as outlined in the documentation: Key-Value Pairs.

In a log processing rule, you can use the KVP operator to transform the necessary key-value pairs into JSON.

For example:

PARSE(content,"'[' JSONTIMESTAMP:timestamp '] ' KVP{'['[^[:]*:key ': ' DATA:value '] '}:kvp '[[' DATA:message ']]'")
|FIELDS_ADD(loglevel:kvp[LOGLEVEL])

Re: Is there a reverse command of fieldsFlatten?

lucassilveira — Thu, 21 Nov 2024 12:34:30 GMT

Perfect, thank you!

Re: Is there a reverse command of fieldsFlatten?

lucassilveira — Thu, 21 Nov 2024 13:21:47 GMT

Nice. I didn't know this feature.