topic How to use the parse command when possible empty value between separators in DQL

How to use the parse command when possible empty value between separators

AurelienGravier — Thu, 17 Oct 2024 20:56:07 GMT

Hello,

I have to extract data from a line of log using a specific format :

provider;id;name;bu;domaine;app;amount;env

My DQL request below works when there is no empty value on my record :

data record(a = "GCP;128773868256;ICDO-KISS;PLFCHAP;PLATE;MYVALUE;DEV;5")
| parse a, ("LD:provider ';' LD:id ';' LD:name ';' LD:bu ';' LD:domaine ';' LD:app ';' LD:amount ';' LD:env")

But it doesn't work if there is an empty value :

data record(a = "GCP;128773868256;ICDO-KISS;PLFCHAP;PLATE;;DEV;5")
| parse a, ("LD:provider ';' LD:id ';' LD:name ';' LD:bu ';' LD:domaine ';' LD:app ';' LD:amount ';' LD:env")

How to manage empty value on parse command please ?

Thank you.

Regards Aurelien

Re: How to use the parse command when possible empty value between separators

Fin_Ubels — Fri, 18 Oct 2024 05:10:39 GMT

Hey @AurelienGravier

For your specific example you can use "Optional Modifier" as documented here. Using your query, that would look something like the below:

data record(a = "GCP;128773868256;ICDO-KISS;PLFCHAP;PLATE;;DEV;5") | parse a, ("WORD:provider ';' WORD:id ';' LD:name ';' WORD:bu ';' WORD:domaine ';' WORD?:app ';' WORD:amount ';' WORD:env")

One thing to note though is that I have change the above to use WORD instead of LD. This is because LD matches to semicolons which means if the value in-between is missing it will just match to the second semicolon and the parsing will fail. For the given example you could use WORD for all fields except for name. For name I cam up with a character class that looks like [0-9a-zA-Z-] and the use of which is documented here but the character class doesn't seem to work. Testing the character class in regex works but within the DQL parse command it doesn't seem to work even though it's supposed to be supported, potentially it requires some more tweaking.

Hope the above helps!

Re: How to use the parse command when possible empty value between separators

AurelienGravier — Fri, 18 Oct 2024 07:26:22 GMT

Hello @Fin_Ubels ,

Thank you for your help, I tried "Optional Modifier" without success because I didn't try to change the LD type.

FYI, with a character group character-group the name field is parsed with success :

data record(a = "GCP;128773868256;;PLFCHAP;PLATE;;DEV;5")
| parse a, ("WORD:provider ';' WORD:id ';' [0-9a-zA-Z-]{1,100}?:name ';' WORD:bu ';' WORD:domaine ';' WORD?:app ';' WORD:amount ';' WORD:env")

Regards Aurélien