cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Log Processing rule limit of line lenght?

r_weber
DynaMight Champion
DynaMight Champion

Has anyone noticed this:
I create a log processing rule to remove parts of the content and only keep relevant data (remove everything after the '[ERROR]' and store it in a new field "message"

The rule works fine when tested, but when applied at runtime it cuts off parts of the content.

r_weber_0-1701075702160.png

 

Here, querying Grail after the processing has been applied (note the cut off "message" field):

r_weber_1-1701075829188.png

Are there any settings/limit I should be aware of for such cleanup rules?

 

Certified Dynatrace Master, Dynatrace Partner - 360Performance.net
3 REPLIES 3

sinisa_zubic
Dynatrace Champion
Dynatrace Champion

Hi @r_weber 

the LD matcher matches per default the first 4096 characters until the next line break. You might have some wired line break. therefore I would suggest to go with the DATA matcher and specify a higher character limit.

But I would suggest that you try the pattern first in notebooks and see if it works the same way as in the processing definition.

fetch logs
| filter <your custom filter for the relevant records>
| parse content, """LD
TIMESTAMP('yyyy-mm-dd HH:mm:ss +S', locale='de') 
LD ('[Error]'|'[Warning]') 
DATA:message"""

 

Best,
Sini

GerardJ
Observer

Hello @r_weber 

I've seen the same behavior with a processing rule that parses a JSON. When I test the rules everything is Ok, but when log is processed, the extracted field seems to be limited to 250 characters.
Have you find any information or a solution ?

r_weber
DynaMight Champion
DynaMight Champion

Hi @GerardJ ,

I solved it as @sinisa_zubic suggested by adding a quantifier to the DATA / LD matcher like this:

USING(INOUT content:STRING) | 
PARSE(content, "
  LD
  TIMESTAMP('yyyy-mm-dd HH:mm:ss +S', locale='de')
  LD
  ('[Error]'|'[Warning]')
  DATA{1,8192}:content
")

 

Hope that helps,
Reinhard

Certified Dynatrace Master, Dynatrace Partner - 360Performance.net

Featured Posts