There can be many reasons why the logs are not visible in Dynatrace; the common reasons are below.
- Check if the One agent running in the host is the supported version/latest version.
(https://docs.dynatrace.com/docs/shortlink/release-notes)
- Check if the Log agent is running in the host; you can check by going to the host page -> Process analysis section- Filter by technology Dynatrace and see if the Log agent is running.
- Check if Log Monitoring is enabled in your Dynatrace environment globally (Dynatrace web UI), or check if Log Monitoring is enabled on a host level (OneAgent CLI). To check if Dynatrace Log Monitoring is enabled globally:
- Go to "Settings > Monitoring > Monitored technologies".
- Find "Log Monitoring" in the list of supported technologies, and select "Edit" (pencil icon).
- Check if "Monitor Log Monitoring" on every host option is enabled.
- Log Monitoring is disabled on that host. Check "Settings -> Technology -> Log Monitoring" page on WebUI.
- Log content access disabled (https://docs.dynatrace.com/docs/setup-and-configuration/dynatrace-oneagent/oneagent-configuration-vi...)
- Improper ingest rules (https://docs.dynatrace.com/docs/observe-and-explore/logs/lma-log-ingestion-via-oa, follow also hints visible on configuration pages on WebUI), especially:
- No ingest rule was created to get the log content. These are for the automatically detected logs. (https://docs.dynatrace.com/docs/observe-and-explore/logs/lma-log-ingestion-via-oa/lma-autodiscovery)... To see if the logs are detected my Dynatrace, the easier way to check the process group page and check the logs tab.
- If the logs are not detected then a custom rule is needed (https://docs.dynatrace.com/docs/shortlink/lma-custom-log-source). For a custom rule to work properly and a log ingestion rule is also needed. (There is now a built-in log ingest rule that you can enable that will ingest all logs from custom log sources. Just enable that rule and you don't have to worry about making multiple rules.)
- Multiple matchers of the same type (do not confuse with multiple values for the same matcher). Remember that you have "AND" operator between matchers, so usually you want max one occurrence of a single attribute type in the scope of a single rule.
- Expecting partial match in a matcher (when actually full match is being done, so to have a partial match you need prefix and suffix wildcards).
- Expecting case insensitive matching (when it is case sensitive, except log source names on Windows).
- A typo, including non-visible characters, or non-ANSI Unicode characters.
- Improper rule order - typically you need more specific closer to the top as they are executed top to bottom and the first catch is decisive
- Improper scope - keep in mind that rules on more specific levels are evaluated earlier (so they are more important) - host before host group, and host group before tenant (https://docs.dynatrace.com/docs/shortlink/lma-log-ingest-rules#configuration-scopes)
- Security rules are violated (https://docs.dynatrace.com/docs/shortlink/lma-custom-log-source#security-rules)
Sample log
[2024-11-28 10:11:29.463 UTC] [/rework/logprocessing/filelogsource.cpp] [info] LGI: / /logs/appl/conc/out/*.txt doesn't meet security rules. (This message was ignored 14 times
- Log Agent does not have access to the file. It might especially happen when a file is on NFS drive on Linux. Then you need to ensure that a user account Log Agent is running on has access to the file (i.e. read privileges to the file, and read with execute to all the directories on the file path) and also enable NFS drive log detection, Settings > Log Monitoring > Advanced log settings
- The file and the folder where the logs reside, dtuser account must have access to it, if not logs wont be ingested.
Sample log
[2024-11-28 01:00:23.630 UTC] [/rework/logprocessing/filelogsource.cpp] [debug] For /opt/logs/tomcat-stdOut.log no file has been found or files are not accessible
- Improper format
- supported encodings include UTF-8 and UTF-16
- No new content after the file has been configured to be sent - Log Agent does not send historical data.
- Auto-detection limits are here: https://docs.dynatrace.com/docs/observe-and-explore/logs/lma-log-ingestion-via-oa/lma-autodiscovery#...
- Log Monitoring default limits are mentioned here (https://docs.dynatrace.com/docs/shortlink/log-monitoring-limits)
- For managed environments, it's also worth checking the cluster events to see if there are any warnings as mentioned here (https://docs.dynatrace.com/docs/shortlink/lmc-troubleshooting)
Hi @noel_david ,
Here an example: for this kind of rules
we have seen that if the process group doesn´t have deployed services
the log isn´t captured. But it is in case the process group has deployed services.
Please, let me know in case my explanation is not clear and it try to explain it better again.
Thanks you so much.
Regards,
Elena.
