topic Extracting key-value pairs from log string into arrays in DQL

Extracting key-value pairs from log string into arrays

Akhil-Jayendran — Mon, 13 Oct 2025 22:51:37 GMT

Hello Team,

I have a log entry in the following format:

This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3.

From this log, I’d like to extract the values of each field and store them in arrays
for example:

AB = [4, 3], CD = [3, 6], and so on.

Has anyone tried implementing something similar or have suggestions on how to achieve this?

I tried this way but not work

data record( content="This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3") | fieldsAdd test=splitString(content, "~") | fieldsAdd test=toString(test) | parse test, "'[' KVP{ '\"' LD:key ':' LD:value '\"' ', '? }:test ']'"

Regards,

Akhil Jayendran

Re: Extracting key-value pairs from log string into arrays

Mohamed_Hamdy — Tue, 14 Oct 2025 06:53:16 GMT

Hi @Akhil-Jayendran ,

I believe the below should work.

data record(content="This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3") | fieldsAdd test = splitString(content, "~") | expand test | filter contains(test, ":") | parse test, "LD:key ':' LONG:value" | summarize values = collectArray(value), by: { key }

Re: Extracting key-value pairs from log string into arrays

Akhil-Jayendran — Tue, 14 Oct 2025 14:52:26 GMT

Hello @Mohamed_Hamdy ,

Appreciate your help, it helped me partially
My end goal is to display the data like this , without summarize any way to get this info

Content	AB	CD	EF	arraymax(AB)	arraymin(AB)	arraymax(CD)	arraymin(CD)
This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3	4,3	3,6	5	4	3	6	3

Regards,
Akhil Jayendran

Re: Extracting key-value pairs from log string into arrays

t_pawlak — Wed, 15 Oct 2025 14:05:26 GMT

Try this:

data record(content = "This is test logs ~AB:4~CD:3~CD:6~EF:5~AB:3") | fieldsAdd parts = splitString(content, "~") | expand parts | filter contains(parts, ":") | parse parts, "LD:key ':' LONG:value" | summarize AB = arrayRemoveNulls(collectArray(if(key == "AB", value, else: NULL))), CD = arrayRemoveNulls(collectArray(if(key == "CD", value, else: NULL))), EF = arrayRemoveNulls(collectArray(if(key == "EF", value, else: NULL))), by: { content } | fieldsAdd `arraymax(AB)` = arrayMax(AB), `arraymin(AB)` = arrayMin(AB), `arraymax(CD)` = arrayMax(CD), `arraymin(CD)` = arrayMin(CD)

you should revice this: