What do log ingest warnings on my records mean? (dt.ingest.warnings)

noel_david · ‎04 Sep 2026

Summary
Problem
Troubleshooting steps
Complete warning reference
The _pipe suffix: what it means
Relationship to SFM events
Common scenarios
What's next
Related reading

Summary

Use the dt.ingest.warnings attribute to identify which individual log records were affected by ingestion limits or processing errors, and understand what each warning value means and how to resolve it.

Problem

Use this article when ingested log records don't look as expected content appears truncated, attributes are missing, timestamps are wrong, or processing rules seem to have no effect and you want to confirm whether the cause is an ingestion limit or a processing error rather than a configuration problem.

dt.ingest.warnings is an array attribute set on individual log records when the ingestion pipeline detects and acts on a limit or processing condition. It is the record-level complement to SFM events: SFM events tell you about the health of log sources and components; dt.ingest.warnings tells you about the health of specific records.

Troubleshooting steps

How to find records with warnings

View all warnings with counts and source

fetch logs
| filter isNotNull(dt.ingest.warnings)
| summarize count(), by:{dt.ingest.warnings, dt.openpipeline.source, log.source}
| limit 100

Find records with a specific warning

fetch logs
| filter matchesValue(dt.ingest.warnings, "content_trimmed")
| limit 100

Important: dt.ingest.warnings is an array type. The filter dt.ingest.warnings == "content_trimmed" will not work. Always use matchesValue() or isNotNull().

Alternative: detect truncation by content length

If dt.ingest.warnings is not available in your environment version, use content length as a proxy:

fetch logs
| filter stringLength(content) == 8192

Or search for the truncation marker appended in some ingest paths:

fetch logs
| filter endsWith(content, "[TRUNCATED]")

Complete warning reference

Content and attribute size warnings

Warning value	When it appears	What to do
`content_trimmed`	Content exceeded the maximum byte size limit at ingest (before processing rules)	Trim content at the sender or shipper before ingesting; reduce log verbosity for high-volume sources
`content_trimmed_pipe`	Content exceeded the maximum byte size limit after OpenPipeline processing rules were applied	Review processing rules that expand content; the final processed content must stay within limits
`attr_count_trimmed`	Number of attributes exceeded the maximum per-record count at ingest	Reduce the number of attributes sent per record; consolidate attributes where possible
`attr_count_trimmed_pipe`	Attribute count exceeded the limit after processing rules were applied	Review processing rules that add attributes; prune attributes not needed for querying or dashboards
`attr_key_trimmed`	At least one attribute key exceeded the 100-byte key size limit	Shorten attribute key names at the sender
`attr_val_count_trimmed`	A multi-value attribute had too many values at ingest	Reduce the number of values in multi-value attributes
`attr_val_count_trimmed_pipe`	A multi-value attribute had too many values after processing rules	Review processing rules that generate multi-value attributes
`attr_val_size_trimmed`	At least one attribute value exceeded the maximum value byte size at ingest	Trim large attribute values at the sender; attribute values must stay within the documented limit
`attr_val_size_trimmed_pipe`	At least one attribute value exceeded the limit after processing rules	Review processing rules that expand attribute values

Timestamp warnings

Warning value	When it appears	What to do
`timestamp_corrected`	The record timestamp was more than 10 minutes in the future; it was reset to the current time	Correct the timestamp at the source or sender; ensure the source system clock is synchronised

Attribute correction warnings

Warning value	When it appears	What to do
`common_attr_corrected`	At least one of the following reserved attributes was corrected: `status`, `loglevel`, or `event.type`	Review the values sent for these attributes; they must conform to the expected format or they will be corrected automatically

Processing rule warnings

Warning value	When it appears	What to do
`processing_batch_timeout`	A batch timeout occurred while executing OpenPipeline log processing rules	Review the processing rules a timeout indicates a rule is taking too long; check for complex regular expressions or large data volumes
`processing_transformer_timeout`	An execution timeout occurred in one of the processing transformers	Simplify the processing rule or transformer logic
`processing_transformer_error`	An execution error occurred in one of the processing transformers	Review the transformer configuration; check for syntax errors or invalid field references
`processing_transformer_throttled`	Execution was throttled in one of the processing transformers	Reduce processing complexity or volume; this indicates resource pressure on the pipeline
`processing_output_record_conversion_error`	An output conversion error occurred for some records during processing	Review the output format configuration of the affected processing rule
`processing_prepare_input_error`	A "prepare input" error occurred in one of the enabled processing rules	Review the input preparation step of the affected rule; check for missing or malformed required input fields

The `_pipe` suffix: what it means

Warning values ending in _pipe indicate the condition occurred after OpenPipeline processing rules were applied to the record, rather than at the point of initial ingestion. This distinction matters for diagnosis:

A warning without _pipe → the record arrived too large or with too many attributes fix the sender.
A warning with _pipe → the record was within limits when it arrived, but a processing rule expanded it beyond limits fix the processing rule.

Relationship to SFM events

dt.ingest.warnings and SFM events are complementary diagnostic tools that operate at different levels:

Tool	Level	What it covers
`dt.ingest.warnings`	Individual record	Which specific records were affected by a limit or processing error, and what happened to them
SFM events (`dt.system.events`)	Log source / component	Whether a source is being discovered, whether a component is healthy, and whether data loss is occurring at the pipeline level

Use dt.ingest.warnings when records arrive but don't look right. Use SFM events when records don't arrive at all or a component is unhealthy.

Common scenarios

"My log content is cut off"

Query for content_trimmed or content_trimmed_pipe. Confirm with stringLength(content). → If content_trimmed is present: trim at the sender. → If content_trimmed_pipe is present: review the OpenPipeline processing rule that expands content.

"An attribute is missing or shorter than expected"

Query for attr_val_size_trimmed or attr_val_size_trimmed_pipe. → If present: the attribute value exceeded the byte limit. Trim at the sender (_trimmed) or in the processing rule (_pipe).

"Records appear under the wrong time"

Query for timestamp_corrected. → If present: the source timestamp was more than 10 minutes in the future. Verify the source system clock.

"A processing rule doesn't seem to be working"

Query for processing_transformer_error, processing_transformer_timeout, or processing_batch_timeout. → If present: the rule encountered an error or timeout. Review the rule configuration.

"Records arrived but `dt.ingest.warnings` is absent"

The record was processed without any limit being applied. The issue is not an ingestion limit investigate the configuration (ingest rule, OpenPipeline routing, bucket assignment) instead.

What's next

Opening a support case with below details

DQL query result showing affected dt.ingest.warnings values and counts
Sanitized sample record showing the full dt.ingest.warnings array and the affected fields
OpenPipeline processing rule configuration (sanitized), if _pipe warnings are present
Source system and ingestion method (OneAgent, API, OpenTelemetry, Fluent Bit, etc.)
Applicable OneAgent, ActiveGate, or Operator version