topic Re: unable to extract json from logs and parse it in DQL

unable to extract json from logs and parse it

ramchandgadde — Wed, 16 Jul 2025 15:45:18 GMT

Below is my DQL query and how to extract json and parse the json? Earlier I was using DQL parse command not fetching all data.

data record(content="[INFO][EAP_WORKDAY][DYNATRACE] 2025-07-16 12:45:23.999 Merged_Response [{'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16076_9885bc5832094b06bfc2572009db68d9', 'Process_Type': None, 'Status': 'ERROR', 'Start_Date_and_Time': None, 'WorkdayID': None, 'End_Date_and_Time': '00:00:00', 'Acc_Center_Batch_ID': '16076'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16027_4c323fff6e354e76927f38dd8b9f97b9', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:18:14.937240Z', 'WorkdayID': None, 'End_Date_and_Time': '00:00:06', 'Acc_Center_Batch_ID': '16027'}, {'Integration_Platform': 'DB_NOTIFIER', 'Integration_Process': 'ATLAS_16027_4c323fff6e354e76927f38dd8b9f97b9', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:19:30.785763Z', 'WorkdayID': None, 'End_Date_and_Time': '00:00:06', 'Acc_Center_Batch_ID': '16027'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16101_7b97e61e32804f7f86c81cd926e7d989', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:20:28.723717Z', 'WorkdayID': None, 'End_Date_and_Time': '00:00:02', 'Acc_Center_Batch_ID': '16101'}, {'Integration_Platform': 'DB_NOTIFIER', 'Integration_Process': 'ATLAS_16101_7b97e61e32804f7f86c81cd926e7d989', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:21:03.272498Z', 'WorkdayID': None, 'End_Date_and_Time': '00:00:01', 'Acc_Center_Batch_ID': '16101'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_15972_328a4276cca3494b9040054cac506b83', 'Process_Type': None, 'Status': 'COMPLETED', 'Start_Date_and_Time': None, 'WorkdayID': None, 'End_Date_and_Time': '00:00:00', 'Acc_Center_Batch_ID': '15972'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_15981_d88dea264ae74fa6a4d0fb238d68d113', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:18:23.535363Z', 'WorkdayID': None, 'End_Date_and_Time': '00:00:03', 'Acc_Center_Batch_ID': '15981'}, {'Integration_Platform': 'DB_NOTIFIER', 'Integration_Process': 'ATLAS_15981_d88dea264ae74fa6a4d0fb238d68d113', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:19:33.355444Z', 'WorkdayID': None, 'End_Date_and_Time': '00:00:04', 'Acc_Center_Batch_ID': '15981'}]")
| filter contains(content, "Merged_Response")
| parse content, "LD 'Merged_Response'JSON:json"

Re: unable to extract json from logs and parse it

StrangerThing — Wed, 16 Jul 2025 17:01:54 GMT

So you actually need to use the JSON_ARRAY parser instead of JSON, but the real issue is with the values that are None. Those are not interpreted as strings and are therefore not valid json.

Re: unable to extract json from logs and parse it

ramchandgadde — Wed, 16 Jul 2025 17:15:08 GMT

I tried to replace None value with '' and updated query with Json Array. but no luck

data record(content="[INFO][EAP_WORKDAY][DYNATRACE] 2025-07-16 12:45:23.999 Merged_Response [{'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16076_9885bc5832094b06bfc2572009db68d9', 'Process_Type': '', 'Status': 'ERROR', 'Start_Date_and_Time': '', 'WorkdayID': '', 'End_Date_and_Time': '00:00:00', 'Acc_Center_Batch_ID': '16076'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16027_4c323fff6e354e76927f38dd8b9f97b9', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:18:14.937240Z', 'WorkdayID': '', 'End_Date_and_Time': '00:00:06', 'Acc_Center_Batch_ID': '16027'}, {'Integration_Platform': 'DB_NOTIFIER', 'Integration_Process': 'ATLAS_16027_4c323fff6e354e76927f38dd8b9f97b9', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:19:30.785763Z', 'WorkdayID': '', 'End_Date_and_Time': '00:00:06', 'Acc_Center_Batch_ID': '16027'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16101_7b97e61e32804f7f86c81cd926e7d989', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:20:28.723717Z', 'WorkdayID': '', 'End_Date_and_Time': '00:00:02', 'Acc_Center_Batch_ID': '16101'}, {'Integration_Platform': 'DB_NOTIFIER', 'Integration_Process': 'ATLAS_16101_7b97e61e32804f7f86c81cd926e7d989', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:21:03.272498Z', 'WorkdayID': '', 'End_Date_and_Time': '00:00:01', 'Acc_Center_Batch_ID': '16101'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_15972_328a4276cca3494b9040054cac506b83', 'Process_Type': '', 'Status': 'COMPLETED', 'Start_Date_and_Time': '', 'WorkdayID': '', 'End_Date_and_Time': '00:00:00', 'Acc_Center_Batch_ID': '15972'}, {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_15981_d88dea264ae74fa6a4d0fb238d68d113', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:18:23.535363Z', 'WorkdayID': '', 'End_Date_and_Time': '00:00:03', 'Acc_Center_Batch_ID': '15981'}, {'Integration_Platform': 'DB_NOTIFIER', 'Integration_Process': 'ATLAS_15981_d88dea264ae74fa6a4d0fb238d68d113', 'Process_Type': 'DB', 'Status': 'COMPLETED', 'Start_Date_and_Time': '2025-07-16T07:19:33.355444Z', 'WorkdayID': '', 'End_Date_and_Time': '00:00:04', 'Acc_Center_Batch_ID': '15981'}]")
| filter contains(content, "Merged_Response")
| parse content, "LD 'Merged_Response'JSON_ARRAY:json"
|fields json

Re: unable to extract json from logs and parse it

StrangerThing — Wed, 16 Jul 2025 17:21:46 GMT

Sorry forgot to mention there's a space after Merged_Response so your parsing string should look like this: "LD 'Merged_Response' SPACE JSON_ARRAY:json"

Re: unable to extract json from logs and parse it

ramchandgadde — Wed, 16 Jul 2025 17:33:51 GMT

Hi, I tried with space - still facing same issue.

Re: unable to extract json from logs and parse it

ramchandgadde — Wed, 16 Jul 2025 17:50:52 GMT

to simplify , I removed array and keeping content small. the problem unable extract json it is always coming null

data record(content="[INFO][EAP_WORKDAY][DYNATRACE] 2025-07-16 17:30:24.173 Merged_Entry {'Integration_Platform': 'DB_BATCH', 'Integration_Process': 'ATLAS_16076_9885bc5832094b06bfc2572009db68d9', 'Process_Type': '', 'Status': 'ERROR', 'Start_Date_and_Time': '', 'WorkdayID': '', 'End_Date_and_Time': '00:00:00', 'Acc_Center_Batch_ID': '16076'}")
| filter contains(content, "Merged_Entry")
| parse content, "LD 'Merged_Entry' Json:json"
|fields json

Re: unable to extract json from logs and parse it

StrangerThing — Wed, 16 Jul 2025 19:30:39 GMT

So it needs to be proper json in order to parse. You'll need to replace all the single quotes with double quotes. Your parse command was still missing the space. It will work with this after you change to proper json and double quotes: LD 'Merged_Entry' SPACE JSON:json

Re: unable to extract json from logs and parse it

GerardJ — Thu, 17 Jul 2025 11:49:09 GMT

Hello
As said in this other post from @ramchandgadde , you can use the JSON_ARRAY in non-strict mode to parse such array not following JSON specification

| parse content, """LD "Merged_Response " JSON_ARRAY(strict=false):array"""