topic Re: Parsing the Logs in DQL

Parsing the Logs

Agoyals — Wed, 24 Sep 2025 09:44:07 GMT

I am bit in struggle of parsing the logs.

Example log patten
<servername>.taskmanager.<servername>:<digits>-1333b8.Status.Shuffle.Netty.UsedMemorySegments: 0
<servername>.taskmanager.<servername>:<digits>-1333b8.Status.Shuffle.Netty.UsedMemory: 1

basically I want to extract metric_name and metric_value
Metric_name
Shuffle.Netty.UsedMemorySegments
Shuffle.Netty.UsedMemory

Metric Value
0
1

I tried hard coded values which is working but want dynamic as servername and some other parts are varies.
| fieldsadd replaceString(content,"<servername>.taskmanager.<servername>:<digits>-1333b8.Status.", ""), alias: keyvalue
| parse keyvalue, """LD:metric_name":" LD:metric_value"""

Re: Parsing the Logs

t_pawlak — Wed, 24 Sep 2025 10:54:13 GMT

Hi,
You can parse everything after .Status. up to the colon as the metric_name, and then parse the value after the colon as the metric_value.
Try this:

fetch logs | filter contains(content, ".Status.") | parse content, "LD:ignore '.Status.' STRING:metric_name ':' INT:metric_value" | fields metric_name, metric_value

or If values can be floating-point numbers, use:

fetch logs | filter contains(content, ".Status.") | parse content, "LD:ignore '.Status.' STRING:metric_name ':' DOUBLE:metric_value" | fields metric_name, metric_value

Re: Parsing the Logs

Agoyals — Wed, 24 Sep 2025 11:45:08 GMT

Thanks for this is not working

Re: Parsing the Logs

Agoyals — Wed, 24 Sep 2025 11:59:41 GMT

Actually with slight modification it worked
| parse content, """LD:ignore".Status."LD:metric_name":"LD:metric_value"""