Extracting key-value pairs from log string into arrays

[Akhil-Jayendran]

Hello Team,

I have a log entry in the following format:

This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3.

From this log, I’d like to extract the values of each field and store them in arrays
for example:

AB = [4, 3], CD = [3, 6], and so on.

Has anyone tried implementing something similar or have suggestions on how to achieve this?

I tried this way but not work 

data record( content="This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3")
| fieldsAdd test=splitString(content, "~")
| fieldsAdd test=toString(test)
| parse test, "'[' KVP{ '\"' LD:key ':' LD:value '\"' ', '? }:test ']'"

 Regards,

Akhil Jayendran

[Mohamed_Hamdy · Accepted solution]

Hi @Akhil-Jayendran ,

I believe the below should work.

data record(content="This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3")
| fieldsAdd test = splitString(content, "~")
| expand test
| filter contains(test, ":")
| parse test, "LD:key ':' LONG:value"
| summarize values = collectArray(value), by: { key }

[Akhil-Jayendran]

Hello @Mohamed_Hamdy ,

Appreciate your help, it helped me partially 
My end goal is to  display the data like this , without summarize any way to get this info 

Content	AB	CD	EF	arraymax(AB)	arraymin(AB)	arraymax(CD)	arraymin(CD)
This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3	4,3	3,6	5	4	3	6	3

Regards,
Akhil Jayendran

[t_pawlak · Accepted solution]

Try this:

data record(content = "This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3")
| fieldsAdd parts = splitString(content, "~")
| expand parts
| filter contains(parts, ":")
| parse parts, "LD:key ':' LONG:value"
| summarize
    AB = arrayRemoveNulls(collectArray(if(key == "AB", value, else: NULL))),
    CD = arrayRemoveNulls(collectArray(if(key == "CD", value, else: NULL))),
    EF = arrayRemoveNulls(collectArray(if(key == "EF", value, else: NULL))),
  by: { content }
| fieldsAdd
    `arraymax(AB)` = arrayMax(AB),
    `arraymin(AB)` = arrayMin(AB),
    `arraymax(CD)` = arrayMax(CD),
    `arraymin(CD)` = arrayMin(CD)

you should revice this: