Extract single line CSV data into multiple columns

[andre_vdveen]

Hi, I'm trying to get the data in a CSV log file split into separate columns, but I'm stuck with DQL/DPL.
Any help would be super appreciated!

Also, I assume I'd need a metric for each column if I want to get the values onto a dashboard, right?
Each value separated by a comma, needs to fall into the columns below.

Here's the raw data - the masked text goes with 'Claims' or 'Batch Claims' and is called the 'Component' in the example above:

[andre_vdveen · Accepted solution]

Figured the 1st part out, with some help:

parse (content, "(LD:dup_timeStamp LD:component FLOAT:AvgTime INT:NoOfClaims INT:WorstTime INT:Time_Outs INT:Exceptions FLOAT:Standin )(fs=',')")

[y_buccellato]

I'm in a similar situation.
Kinda surprise that with all the DQL power you cannot auto parse a field which contains multiple value divided by comma ",".
My love for excel still persist 😄

[36Krazyfists]

So, you could potentially do this with a sophisticated enough DPL query, however, the main problem is that you have to know the CSV headers ahead of time. 

For example, I have tons of pipelines that auto-parse JSON and XML logs data, but that's because the JSON/XML records contain the key names and the values in each record.  With CSV, only the first line of a log file would potentially have the column names, and since the log parsing is done one line at a time, it has no way to know about the headers defined in some other record earlier that day or week, so you would have to build a DPL parser that defines the name of each item.