Log with number/hash character in the file name not uploaded

[gilles_tabary]

Hello.

I am on managed 1.274.157.20230908-165051 + OneAgent v1.267.155.20230629-142023+. We moved to log Classic kinda recently.

I have a process group with Dynatrace UI showing auto-detected log files : 

...
/myFSLocation/var/log/tomcat/abcd2/abcd2-daily.log
/myFSLocation/var/log/tomcat/abcd2/abcd#-daily.log (File is not included in log monitoring, some log records may be missing)
...

• I double checked /myFSLocation/var/log/tomcat/abcd2/abcd2-daily.log does comply with dynatrace monitorability prerequisit, including being appended at least each minute. 
• FYI: we have some other logs for which uploading does work. Like for /myFSLocation/var/log/tomcat/abcde/abcde-daily.log (note : no number in log file name).
• FYI : the "Maximum ingest of log events per minute" is high enough.
• Though, we don't get any content being uploaded for this abcd2-daily.log log. I mean zero content, zero lines in.

Could we be exposed to an issue with log rotation detection (https://www.dynatrace.com/support/help/shortlink/log-rotation-patterns) ? Any one experienced this ? Any idea ?

Regards.

--

(My private internal refs: dynt request: 217427 - jira: DEVOPS-14282)

[gilles_tabary]

I observe, when I add an extra rule in "Custom log source configuration" to tell Dynatrace /myFSLocation/var/log/tomcat/abcd2/abcd2-daily.log is a log, then it at last get's uploaded.

But not associated to its PG nor itS PGI. Which is strange, because when I watch through the PG, the tabs "Logs" shows the log file alright. If I add a PG context in the "Custom log source configuration" rule then it's contextualised all-right with PG and PGI.

Yet all this requires per log / per PGI définitions : quite some qork ! Ans not durable : if a PG-ID changes : all gone ! 🙂

Any ideas.

Regards.

[Ingrida]

Hi @gilles_tabary ,

I am a bit puzzled here, your title is "Log with number/hash character in the file name not uploaded" but you are talkign about /myFSLocation/var/log/tomcat/abcd2/abcd2-daily.log not being recognized. so, what is the problem? file name with hashes? numbers, normal file but not being identified?

We are using logging quite heaviliy and if we are absolutelly stuck we used to log to oneagent logs (quite verbose, but only source of information so far).

Something we faced and solved so far were:

- logs not collected due symlinks used (some symlinks are not acepted in DT). Normally we were able to see it as path to logs was different on autotetected logs page as one you would see on machine.

- logs not collected due wrong match rules. You have to list all combinations (log with numbers, log without numbers, etc.) in one matcher, as inside a matche you habe OR, between matchers you have AND

BR,

Ingrida

[gilles_tabary]

Hello.

The problem is : I need /myFSLocation/var/log/tomcat/abcd2/abcd2-daily.log to be uploaded. I am baffled. We lost weeks of log monitoring. We spent weeks on this issue 217427, with no clear solution. Please refer to this ticket for details as it is quite time consuming to give over and over the same context. 🙂

• This log is auto-detected fine
• Yet when I add a "Log storage configuration" rule (full explicit path) : it does not get uploaded.
• I found a hack : add in "Custom log source configuration" an over-specifying rule with same absolute path + PG contextualisation (with the added risk of PG ID changing some days)

It was suggested to specify a "Log storage configuration" rule with /myFSLocation/var/log/tomcat/abcd2/abcd#-daily.log 

• then it looks abcd2-daily.log content is uploaded
• yet abcd2-daily.log is reported in the UI as not monitored : confusing given that's the log file which does exist on the FS, and given tha's the lof file we want to monitore
• abcd#-daily.log is reported as monitored : confusing as there is no such file on the FS
• abcd#-daily.log is considered as a new source, leading to log event history split from from abcd2-daily.log source

FYI : I have some log uploading working fine with non digit including file-name like /myFSLocation/var/log/tomcat/abcde/abcde-daily.log

Regards.

[gilles_tabary]

I still have to clean / clear solution to this.