Conditional Parsing

[mario_rwwa]

Currently I'm dealing with a logfile which outputs different data depending on the action which has occurred, one parse statement cannot handle all the options so I have multiple parse statements.

I need to combine it all into one table at the end, but the only option I've found so far is via multiple fetch & appends, which feels very inefficient and clunky. Is there are way to streamline this sort of query?

As an example:

fetch logs, from: -3d
|  filter dt.host_group.id == "myTag"
|  filter matchesPhrase(content, "myFirstTextToMatch")
| parse content, "DATA blah blah blah parse out fields here"
| append  [fetch logs, from: -3d
  |  filter dt.host_group.id == "myTag"
  |  filter matchesPhrase(content, "mySecondTextToMatch")
  |  parse content, "DATA blah blah blah parse out second pattern fields here"
    | append  [fetch logs, from: -3d
      |  filter dt.host_group.id == "myTag"
      |  filter matchesPhrase(content, "myThirdTextToMatch")
      |  parse content, "DATA blah blah blah parse out third pattern fields here"
... and so on

Is there a way to use a conditional operator here maybe?

ie, IF matches FirstText then parse using First pattern ELSE IF matches SecondText parse using Second pattern...

[ChadTurner]

@mario_rwwa - were you able to get a solution for this?

[marco_irmer]

Rather than the conditional parsing, you might consider parsing at ingest time instead so that the parsing can be removed from the query entirely. This can be advantageous, because you can make the parsing rules conditional to each incoming pattern. Then when you go to query, you just query the pre-parsed fields. 

If that's not feasible, the other approach is to use the fieldsAdd command, combined with the if and parseAll functions to accomplish additional parsing. You'll have to do some array flattening afterwards as well, since the parseAll function will output its result into a single field as an array.

[Emm4nuel · Accepted solution]

Hi Mario,

Option 1: using the parse function with if statement

data record(content = "This is a string with IP: --"), 
record(content = "This is a string with UUID: --")
 | fieldsAdd res = if(stringLength(content) == 37, parse(content, "LD IP:out"), else: parse(content, "LD UUIDSTRING:out"))

Option 2: using parsingPrerequisite

data record(content = "This is a string with IP: --"), 
record(content = "This is a string with UUID: --") 
| parse content, "LD IP:out", parsingPrerequisite: stringLength(content) == 37 
| parse content, "LD UUIDSTRING:out", parsingPrerequisite: stringLength(content) == 64

Option 3 (worst): just hope that at least one pattern works:

data record(content = "This is a string with IP: --"), 
record(content = "This is a string with UUID: --") 
| parse content, "LD IP:out", preserveFieldsOnFailure: true 
| parse content, "LD UUIDSTRING:out", preserveFieldsOnFailure: true

 As of now there is no else if option available. 

Credits to: Maxim Pokrovskiy