Troubleshooting missing logs

Kacper · ‎02 Jun 2026

Troubleshooting missing logs

Summary

This article helps you troubleshoot situations where logs you expect to be ingested by Log Module are missing from Dynatrace. It applies to Log Module deployed via OneAgent or Dynatrace Operator on both SaaS and Managed environments, and covers file-based logs on Linux and Windows as well as Kubernetes container logs.

Problem

Logs you expect to see in Dynatrace are not appearing, appearing only partially, or appearing without the metadata you expect. Common indicators include:

A log file is being actively written on the host, but no records appear in the Logs viewer.
Some records appear, but with consistent gaps for specific intervals.
Logs appear for some sources or severities but not others.
Logs are collected but lack Kubernetes metadata (namespace, pod, workload).
Logs appear with timestamps that are significantly behind the current time.

The cause can range from a missing or misconfigured rule to a known platform limitation. Follow the troubleshooting steps below to narrow it down.

Troubleshooting steps

Pre-flight checklist

Before investigating a specific scenario, verify all of the following.

#	Check	How to verify
1	A log storage rule exists that covers the target source	Settings > Collect and capture > Log Monitoring > Log storage configuration
2	OneAgent is running on the affected host	Infrastructure & Operations > Hosts
3	The log file is not compressed at the time of scan	Check the file extension at the moment; logs should be read
4	App log content access is turned on on the host	Run `oneagentctl --get-app-log-content-access` on the host. If the output is `false`, turn it on with `oneagentctl --set-app-log-content-access=true`
5	Log entries were written after monitoring was configured	Log Module does not backfill data written before a log source was turned on — configure monitoring first, then wait for or reproduce new log entries

For Kubernetes deployments, also check:

#	Check	How to verify
K1	`.spec.logMonitoring` is present in the DynaKube	`kubectl get dynakube -n dynatrace -o jsonpath='{range .items[*]}{.metadata.name}{"\n\t.spec.logMonitoring: "}{.spec.logMonitoring}{"\n\t.spec.metadataEnrichment: "}{.spec.metadataEnrichment}{"\n"}{end}'`
K2	Log Module DaemonSet pods are running on every affected node	`kubectl -n dynatrace get pods -o jsonpath='{range .items[*]}{.metadata.ownerReferences[0].name}{"\t"}{.spec.nodeName}{"\n"}{end}' \| grep -E 'logmonitoring\|oneagent'`
K3	No conflicting exclude rules at a higher scope override your include rule	Settings > Collect and capture > Log Monitoring > Log storage configuration — review all scopes

Is your scenario supported?

If the checklist passes and logs are still missing, check whether your situation is a known limitation.

Scenario	Support	Notes
File-based logs on Linux/Windows (local filesystem)	✅ Supported	Path must be accessible to the OneAgent process
Kubernetes container stdout/stderr via native Kubernetes Log monitoring	✅ Supported	Requires `.spec.logMonitoring` in DynaKube
Auto-discovered log sources with rotation (uncompressed)	✅ Supported	Rotation detected automatically via inode tracking
Custom log sources with rotation (uncompressed)	✅ Supported	Requires `#` wildcard in path pattern
Compressed rotated files (`.gz`, `.bz2`, etc.)	❌ Not supported	Compressed files can't be read
Log rotation with directory change	❌ Not supported	Rotated files moved to a different directory break tracking
Queue-style / circular-buffer sources	❌ Not supported	Log Module assumes append-only files
Copy-and-truncate rotation where the original is truncated before reading completes	❌ Limitation	Content written before truncation may not be fully read
Kubernetes container logs collected via a custom log source (`/var/log/pods/…`)	❌ Not recommended	Results in host-level logs without Kubernetes metadata
AWS Fargate host-level logs	❌ Not supported	Fargate nodes are not accessible to OneAgent
ECS tasks using a non-`json-file` Docker logging driver	❌ Not supported	Only `json-file` produces files that the Log module can read
Log entries with timestamps too far in the past	❌ Not supported	Entries are dropped if the timestamp predates ingestion start by more than 15 minutes (24-hour grace window after)
Log entries with timestamps more than 10 minutes in the future	❌ Not supported	Entries are not dropped but re-timestamped to the processing time

Resolution

Log source detected, but no data — storage rule not matching

Symptoms

A log source is known to exist on the host, but no records appear in the Logs viewer.
Log Module self-monitoring shows log.source.ingest_status = "Not ingested" while log.source.file_status = "FILE_STATUS_OK".

Resolution

Go to Settings → Collect and capture > Log Monitoring > Log storage configuration.
Select the scope that matches your host, host group, or Kubernetes cluster.
Add a rule that matches the target path and set the action to Include in storage.
Verify that no rule placed above it has a broader exclusion that would match first.

Keep these rule behaviors in mind:

Evaluation order: Rules run from top to bottom, and the first match wins. A broad exclude rule above a specific include rule will silently block ingestion. Review the full list, not only the most recently added rule.
Scope priority: Host scope is evaluated before host group, and host group before environment. A more specific scope takes precedence.
Propagation: Changes propagate to agents within ~90 seconds.

Common rule configuration mistakes

Mistake	Detail
Expecting a partial match in the log content matcher	The log content matcher performs a full match. To match a substring, add wildcard prefix and suffix: `exception`
Multiple matchers of the same attribute type in one rule	Matchers within one rule are combined with AND. Two log source matchers means both must match the same source simultaneously, which can never be true. Use one matcher per attribute type.
Case sensitivity	All matchers are case-sensitive. Exception: log source name matching is case-insensitive on Windows.
Wrong log source name	Auto-discovered sources have numeric parts of the filename replaced with `#` (e.g., `app-20260101.log` → `app-#.log`). Kubernetes container logs appear as Container Output. Custom log source names are exactly as defined — they are not derived from the file path.
Expecting the file path to be the log source name	Log ingest rules match against the log source name, not the raw file path. These differ for auto-discovered sources.

Specific log levels missing

Symptoms

ERROR and WARN records from a log source are visible; INFO and DEBUG are absent.
The log source itself is detected, and the active file is read normally.

Resolution

Go to Settings → Collect and capture → Log Monitoring → Log storage configuration.
Inspect rules at every applicable scope: environment, host group, host, and — for Kubernetes — cluster and namespace levels.
Look for any rule that specifies a severity matcher. A rule that includes only ERROR and WARN will drop INFO entries before they reach Grail.
Add or modify the relevant rule to include the desired severity levels.

Severity filtering can also occur further down the pipeline — check Open Pipeline rules, Classic DPP pipeline rules, and any technology packages that may apply filters.

Entries dropped — timestamp out of bounds

Symptoms

Fewer records appear than expected; gaps exist for specific intervals.
Comparing host logs with Dynatrace logs reveals entries that were generated but never ingested.

Cause (brief)

Dynatrace enforces timestamp bounds on ingested entries. Entries with timestamps more than 15 minutes before ingestion started are dropped (with a 24-hour grace window after start). Entries with timestamps more than 10 minutes in the future are re-timestamped to processing time.

A common trigger: the log line contains a date-like string that is not the entry's own timestamp — for example, a business period reference (Begin Time: 2025-03-23 08:15:00) or a JSON field, such as "eventtime": "2025-03-23T08:15:00Z" appearing before the actual log entry time. Log Module may select that value as the timestamp.

Resolution

For plain-text logs: Define a timestamp pattern rule under Settings > Collect and capture > Log Monitoring > Timestamp configuration that includes the prefix text preceding the real timestamp. This anchors the pattern so Log Module matches only the intended timestamp position and ignores other date-like strings.

For JSON logs, choose one of:

Reorder the JSON structure so that a supported timestamp key (such as timestamp) appears before the misleading field.
Turn off JSON parsing for this log source and define a prefix-anchored timestamp pattern rule as for plain-text logs. Note: JSON fields will no longer be automatically extracted as log attributes.
Define a custom timestamp key under Settings → Collect and capture → Log Monitoring → Timestamp configuration that explicitly names the field to use as the log timestamp.

Entries are missing after log rotation

Symptoms

Logs appear normally most of the time, but records written just before or during a log rotation event are absent.
The issue applies to a custom log source. Auto-discovered sources detect rotation automatically.

Resolution

The custom log source path is most likely missing the # wildcard at the rotation identifier position. The # wildcard matches zero or more characters, so it covers both the active file and rotated counterparts.

Active file	Rotated file	Correct pattern
`cas.log`	`cas-20260203.log`	`cas#.log`
`app.log`	`app.log.1`	`app.log`
`app.log`	`app-20260101.log`	`app#.log`

Go to Settings → Collect and capture → Log Monitoring → Custom log sources.
Update the path pattern to include # at the position where the rotation identifier appears. Example: G:\Logs\app\VisecaOneSA.log → G:\Logs\app\VisecaOneSA#.log.
Confirm that the active filename still matches the new pattern.

New rotated files may take up to 5 minutes to be detected after the change.

For a deeper guide to rotation issues, see Troubleshooting log rotation issues with Log Module.

Custom log source blocks Kubernetes enrichment

Symptoms

Kubernetes pod logs are collected and visible in the Logs viewer, but they're associated with the host only, and not with any Kubernetes entity (pod, workload, namespace).
Filtering logs by namespace or workload in the Kubernetes app returns no results for the affected pods.
Custom log source rules targeting /var/log/pods/ or /var/log/containers/ are present in the configuration.

Cause

When a custom log source targets /var/log/pods/ or /var/log/containers/, the Log module treats those files as plain host-level log files and does not apply Kubernetes enrichment. Only logs collected through native Kubernetes container log monitoring receive k8s.* metadata automatically.

Resolution

Remove custom log source rules that target /var/log/pods/ or /var/log/containers/.
Ensure .spec.logMonitoring is present in the DynaKube resource. See the Kubernetes troubleshooting guide for full setup details.
Configure log storage rules at the Kubernetes cluster scope under Settings > Collect and capture > Log Monitoring > Log storage configuration.

After removing the conflicting custom log source rules, logs collected via native container monitoring receive full Kubernetes enrichment automatically.

Container logs missing — unsupported logging driver

Symptoms

Container logs are not collected from ECS tasks or Docker-based containers.
OneAgent detects the container and collects process metrics, but no log records appear.
The agent log contains FILE_STATUS_NOT_EXIST information for the container group instance.

Diagnosis

On the Docker host, check the logging driver:

bash

docker inspect -f '{{.HostConfig.LogConfig.Type}}' <container_id>

If the result is notjson-file, the container uses an unsupported driver. Only the json-file the Docker logging driver produces files that the Log Module can read. Drivers such as awslogs, splunk, and fluentd route output to external targets and write no local file.

Resolution

Switch the container's logging driver to json-file.
If OneAgent monitoring mode is set to infrastructure-only, switch to full stack monitoring. Application log content access is turned off by default in infrastructure-only mode.

Logs appear late — application writes historical timestamps

Symptoms

Log records appear in Dynatrace, but the timestamp shown is significantly behind the current time (1–2 hours or more).
Querying a recent time window misses records that appear when the window is extended further back.

Cause (brief)

The application is writing log entries with timestamps that reflect the business event time, not the wall time at which the entry was written. A common example: a service that aggregates metrics at 1-minute intervals and writes summary entries with the period start time, which may be 30–90 minutes in the past when the entry is flushed to disk.

This is not a collection error. The records are ingested and stored correctly.

Resolution

For applications that intentionally log historical data:

Extend the query time range to cover the historical period described by the log content.
Sort by ingestion order rather than content timestamp to see records in the sequence they arrived.

If the historical timestamp is undesirable:

Review whether the application can be changed to log the current wall time.
Alternatively, configure Settings > Collect and capture > Log Monitoring > Timestamp configuration to use a different pattern or field for that log source.

Log file not auto-discovered

Symptoms

No log source appears in the Log Module self-monitoring dashboard for the expected file.
The file exists on the host and is actively written, but is invisible to the Log module.

Cause

Autodiscovery detects only files that meet all of the following criteria:

Kept open by an important process (a process monitored in full stack mode; for infrastructure-only hosts, only system-level processes qualify).
Exists for at least 1 minute.
Updated within the last 7 days.
File path contains a /log/ or /logs/ directory segment, or the filename has a .log or _log suffix.

Resolution

Scenario	Resolution
Log file written by an unimportant process	Make the process important via declarative process grouping, or define a custom log source under Settings → Collect and capture → Log Monitoring → Custom log sources
Log file on a network filesystem (NFS/SMB) on Linux	Enable Search for logs on mounted network drives under Settings → Collect and capture → Log Monitoring → Advanced log settings
File path or name does not match naming conventions	Define a custom log source with an explicit path pattern
Log file written directly inside a container filesystem	Container-internal files are not visible to the Log module. Redirect output to stdout/stderr instead, or define a custom log source pointing to the host-visible path under `/var/log/pods/` — but see the Kubernetes enrichment trade-off above
Windows Event Log	Windows Event Logs are not discovered as file-based sources. Define a Windows Event Log custom source type
Autodiscovery limit reached	Log Module discovers up to 200 log sources per process group instance. If the limit is reached, define a custom log source for the missing file

Security rules are blocking ingestion

Symptoms

App log content access is allowed, a custom log source is configured, and no storage rule blocks the source, yet no records appear, and the source does not appear in the Log module self-monitoring dashboard.
The file path is outside standard log directories or uses a non-standard extension.

Cause

Log Module enforces built-in security rules that restrict which file paths may be ingested. A path that fails the security check is silently blocked.

By default, a file is permitted when at least one of the following is true:

The filename has a .log or .txt extension (rotation separators ., -, _ are allowed, e.g., app.log.1 or app-2026-01-01.log).
The file resides in a directory named log, logs, applogs, logfile, or logfiles.
The filename is catalina.out.

Paths blocked by default include system directories (/etc, /boot, /proc, /dev, /bin, /sbin), hidden directories, and paths containing .ssh or .pem.

Resolution

Use the Dynatrace Log Security Rules Checker to verify whether your log paths pass the security rule check and to generate a custom rules configuration file if exceptions are needed.

To add custom rules manually, create a JSON configuration file in the Log Module config directory:

Platform	Path
Linux	`/var/lib/dynatrace/oneagent/agent/config/logmodule/`
Windows	`%PROGRAMDATA%\dynatrace\oneagent\agent\config\logmodule\`

Rules are evaluated top-to-bottom; the first matching rule wins. Custom rules in this directory take precedence over built-in rules. Relaxing the rules only affects custom log sources. Restricting rules affects both auto-discovered and custom sources.

What's next

For rotation-specific symptoms, see Troubleshooting log rotation issues with Log Module.
For ingest delay (logs arrive but late), see Troubleshooting log ingest delay.
For boundary issues (multi-line entries split incorrectly, or entries merged), see Troubleshooting log record boundary detection.
For Kubernetes metadata enrichment issues (missing k8s.* attributes, PodListFuture errors), see the Kubernetes troubleshooting guide.

Troubleshooting missing logs
Table of contents
Summary
Problem
Troubleshooting steps
Resolution
What's next