https://community.dynatrace.com/t5/Troubleshooting/What-do-log-ingest-warnings-on-my-records-mean-dt-ingest/ta-p/303795 <P><LI-TOC indent="15" liststyle="disc" maxheadinglevel="2"></LI-TOC></P> <DIV class="lia-message-template-content-zone"> <P>&nbsp;</P> <H2>Summary</H2> <P><SPAN>Use the </SPAN><CODE>dt.ingest.warnings</CODE><SPAN> attribute to identify which individual log records were affected by ingestion limits or processing errors, and understand what each warning value means and how to resolve it.</SPAN></P> <P>&nbsp;</P> <H2>Problem</H2> <P>Use this article when ingested log records don't look as expected&nbsp; content appears truncated, attributes are missing, timestamps are wrong, or processing rules seem to have no effect and you want to confirm whether the cause is an ingestion limit or a processing error rather than a configuration problem.</P> <P><CODE>dt.ingest.warnings</CODE> is an array attribute set on individual log records when the ingestion pipeline detects and acts on a limit or processing condition. It is the record-level complement to SFM events: SFM events tell you about the health of log <STRONG>sources and components</STRONG>; <CODE>dt.ingest.warnings</CODE> tells you about the health of specific <STRONG>records</STRONG>.</P> <P>&nbsp;</P> <H2>Troubleshooting steps</H2> <H3>How to find records with warnings</H3> <H4>&nbsp;</H4> <H4>View all warnings with counts and source</H4> <DIV class="dutmcw0"> <PRE class="dutmcw2"><CODE class="language-dql">fetch logs | filter isNotNull(dt.ingest.warnings) | summarize count(), by:{dt.ingest.warnings, dt.openpipeline.source, log.source} | limit 100 </CODE></PRE> </DIV> <H4>&nbsp;</H4> <H4>Find records with a specific warning</H4> <DIV class="dutmcw0"> <PRE class="dutmcw2"><CODE class="language-dql">fetch logs | filter matchesValue(dt.ingest.warnings, "content_trimmed") | limit 100 </CODE></PRE> </DIV> <BLOCKQUOTE> <P><STRONG>Important:</STRONG> <CODE>dt.ingest.warnings</CODE> is an <STRONG>array type</STRONG>. The filter <CODE>dt.ingest.warnings == "content_trimmed"</CODE> will <STRONG>not</STRONG> work. Always use <CODE>matchesValue()</CODE> or <CODE>isNotNull()</CODE>.</P> </BLOCKQUOTE> <H4>&nbsp;</H4> <H4>Alternative: detect truncation by content length</H4> <P>If <CODE>dt.ingest.warnings</CODE> is not available in your environment version, use content length as a proxy:</P> <DIV class="dutmcw0"> <PRE class="dutmcw2"><CODE class="language-dql">fetch logs | filter stringLength(content) == 8192 </CODE></PRE> </DIV> <P>Or search for the truncation marker appended in some ingest paths:</P> <DIV class="dutmcw0"> <PRE class="dutmcw2"><CODE class="language-dql">fetch logs | filter endsWith(content, "[TRUNCATED]") </CODE></PRE> </DIV> <HR /> <H2>Complete warning reference</H2> <H3>&nbsp;</H3> <H3>Content and attribute size warnings</H3> <P>&nbsp;</P> <TABLE> <THEAD> <TR> <TH class="lia-align-center"><FONT size="2">Warning value</FONT></TH> <TH class="lia-align-center"><FONT size="2">When it appears</FONT></TH> <TH class="lia-align-center"><FONT size="2">What to do</FONT></TH> </TR> </THEAD> <TBODY> <TR> <TD><CODE>content_trimmed</CODE></TD> <TD>Content exceeded the maximum byte size limit<SPAN>&nbsp;</SPAN><STRONG>at ingest</STRONG><SPAN>&nbsp;</SPAN>(before processing rules)</TD> <TD>Trim content at the sender or shipper before ingesting; reduce log verbosity for high-volume sources</TD> </TR> <TR> <TD><CODE>content_trimmed_pipe</CODE></TD> <TD>Content exceeded the maximum byte size limit<SPAN>&nbsp;</SPAN><STRONG>after</STRONG><SPAN>&nbsp;</SPAN>OpenPipeline processing rules were applied</TD> <TD>Review processing rules that expand content; the final processed content must stay within limits</TD> </TR> <TR> <TD><CODE>attr_count_trimmed</CODE></TD> <TD>Number of attributes exceeded the maximum per-record count<SPAN>&nbsp;</SPAN><STRONG>at ingest</STRONG></TD> <TD>Reduce the number of attributes sent per record; consolidate attributes where possible</TD> </TR> <TR> <TD><CODE>attr_count_trimmed_pipe</CODE></TD> <TD>Attribute count exceeded the limit<SPAN>&nbsp;</SPAN><STRONG>after</STRONG><SPAN>&nbsp;</SPAN>processing rules were applied</TD> <TD>Review processing rules that add attributes; prune attributes not needed for querying or dashboards</TD> </TR> <TR> <TD><CODE>attr_key_trimmed</CODE></TD> <TD>At least one attribute key exceeded the 100-byte key size limit</TD> <TD>Shorten attribute key names at the sender</TD> </TR> <TR> <TD><CODE>attr_val_count_trimmed</CODE></TD> <TD>A multi-value attribute had too many values<SPAN>&nbsp;</SPAN><STRONG>at ingest</STRONG></TD> <TD>Reduce the number of values in multi-value attributes</TD> </TR> <TR> <TD><CODE>attr_val_count_trimmed_pipe</CODE></TD> <TD>A multi-value attribute had too many values<SPAN>&nbsp;</SPAN><STRONG>after</STRONG><SPAN>&nbsp;</SPAN>processing rules</TD> <TD>Review processing rules that generate multi-value attributes</TD> </TR> <TR> <TD><CODE>attr_val_size_trimmed</CODE></TD> <TD>At least one attribute value exceeded the maximum value byte size<SPAN>&nbsp;</SPAN><STRONG>at ingest</STRONG></TD> <TD>Trim large attribute values at the sender; attribute values must stay within the documented limit</TD> </TR> <TR> <TD><CODE>attr_val_size_trimmed_pipe</CODE></TD> <TD>At least one attribute value exceeded the limit<SPAN>&nbsp;</SPAN><STRONG>after</STRONG><SPAN>&nbsp;</SPAN>processing rules</TD> <TD>Review processing rules that expand attribute values</TD> </TR> </TBODY> </TABLE> <H3>&nbsp;</H3> <H3>Timestamp warnings</H3> <P>&nbsp;</P> <TABLE> <THEAD> <TR> <TH class="lia-align-center"><FONT size="2">Warning value</FONT></TH> <TH class="lia-align-center"><FONT size="2">When it appears</FONT></TH> <TH class="lia-align-center"><FONT size="2">What to do</FONT></TH> </TR> </THEAD> <TBODY> <TR> <TD><CODE>timestamp_corrected</CODE></TD> <TD>The record timestamp was more than 10 minutes in the future; it was reset to the current time</TD> <TD>Correct the timestamp at the source or sender; ensure the source system clock is synchronised</TD> </TR> </TBODY> </TABLE> <H3>&nbsp;</H3> <H3>Attribute correction warnings</H3> <P>&nbsp;</P> <TABLE> <THEAD> <TR> <TH class="lia-align-center"><FONT size="2">Warning value</FONT></TH> <TH class="lia-align-center"><FONT size="2">When it appears</FONT></TH> <TH class="lia-align-center"><FONT size="2">What to do</FONT></TH> </TR> </THEAD> <TBODY> <TR> <TD><CODE>common_attr_corrected</CODE></TD> <TD>At least one of the following reserved attributes was corrected:<SPAN>&nbsp;</SPAN><CODE>status</CODE>,<SPAN>&nbsp;</SPAN><CODE>loglevel</CODE>, or<SPAN>&nbsp;</SPAN><CODE>event.type</CODE></TD> <TD>Review the values sent for these attributes; they must conform to the expected format or they will be corrected automatically</TD> </TR> </TBODY> </TABLE> <H3>&nbsp;</H3> <H3>Processing rule warnings</H3> <P>&nbsp;</P> <TABLE> <THEAD> <TR> <TH class="lia-align-center"><FONT size="2">Warning value</FONT></TH> <TH class="lia-align-center"><FONT size="2">When it appears</FONT></TH> <TH class="lia-align-center"><FONT size="2">What to do</FONT></TH> </TR> </THEAD> <TBODY> <TR> <TD><CODE>processing_batch_timeout</CODE></TD> <TD>A batch timeout occurred while executing OpenPipeline log processing rules</TD> <TD>Review the processing rules&nbsp; a timeout indicates a rule is taking too long; check for complex regular expressions or large data volumes</TD> </TR> <TR> <TD><CODE>processing_transformer_timeout</CODE></TD> <TD>An execution timeout occurred in one of the processing transformers</TD> <TD>Simplify the processing rule or transformer logic</TD> </TR> <TR> <TD><CODE>processing_transformer_error</CODE></TD> <TD>An execution error occurred in one of the processing transformers</TD> <TD>Review the transformer configuration; check for syntax errors or invalid field references</TD> </TR> <TR> <TD><CODE>processing_transformer_throttled</CODE></TD> <TD>Execution was throttled in one of the processing transformers</TD> <TD>Reduce processing complexity or volume; this indicates resource pressure on the pipeline</TD> </TR> <TR> <TD><CODE>processing_output_record_conversion_error</CODE></TD> <TD>An output conversion error occurred for some records during processing</TD> <TD>Review the output format configuration of the affected processing rule</TD> </TR> <TR> <TD><CODE>processing_prepare_input_error</CODE></TD> <TD>A "prepare input" error occurred in one of the enabled processing rules</TD> <TD>Review the input preparation step of the affected rule; check for missing or malformed required input fields</TD> </TR> </TBODY> </TABLE> <HR /> <H2>&nbsp;</H2> <H2>The <CODE>_pipe</CODE> suffix: what it means</H2> <P>Warning values ending in <CODE>_pipe</CODE> indicate the condition occurred <STRONG>after</STRONG> OpenPipeline processing rules were applied to the record, rather than at the point of initial ingestion. This distinction matters for diagnosis:</P> <UL> <LI>A warning <STRONG>without</STRONG> <CODE>_pipe</CODE> → the record arrived too large or with too many attributes&nbsp; fix the sender.</LI> <LI>A warning <STRONG>with</STRONG> <CODE>_pipe</CODE> → the record was within limits when it arrived, but a processing rule expanded it beyond limits fix the processing rule.</LI> </UL> <HR /> <H2>&nbsp;</H2> <H2>Relationship to SFM events</H2> <P><CODE>dt.ingest.warnings</CODE> and SFM events are complementary diagnostic tools that operate at different levels:</P> <TABLE> <THEAD> <TR> <TH class="lia-align-center"><FONT size="2">Tool</FONT></TH> <TH class="lia-align-center"><FONT size="2">Level</FONT></TH> <TH class="lia-align-center"><FONT size="2">What it covers</FONT></TH> </TR> </THEAD> <TBODY> <TR> <TD><CODE>dt.ingest.warnings</CODE></TD> <TD>Individual record</TD> <TD>Which specific records were affected by a limit or processing error, and what happened to them</TD> </TR> <TR> <TD>SFM events (<CODE>dt.system.events</CODE>)</TD> <TD>Log source / component</TD> <TD>Whether a source is being discovered, whether a component is healthy, and whether data loss is occurring at the pipeline level</TD> </TR> </TBODY> </TABLE> <P>Use <CODE>dt.ingest.warnings</CODE> when records arrive but don't look right. Use SFM events when records don't arrive at all or a component is unhealthy.</P> <HR /> <H2>Common scenarios</H2> <H3>"My log content is cut off"</H3> <P>Query for <CODE>content_trimmed</CODE> or <CODE>content_trimmed_pipe</CODE>. Confirm with <CODE>stringLength(content)</CODE>. → If <CODE>content_trimmed</CODE> is present: trim at the sender. → If <CODE>content_trimmed_pipe</CODE> is present: review the OpenPipeline processing rule that expands content.</P> <H3>&nbsp;</H3> <H3>"An attribute is missing or shorter than expected"</H3> <P>Query for <CODE>attr_val_size_trimmed</CODE> or <CODE>attr_val_size_trimmed_pipe</CODE>. → If present: the attribute value exceeded the byte limit. Trim at the sender (<CODE>_trimmed</CODE>) or in the processing rule (<CODE>_pipe</CODE>).</P> <H3>&nbsp;</H3> <H3>"Records appear under the wrong time"</H3> <P>Query for <CODE>timestamp_corrected</CODE>. → If present: the source timestamp was more than 10 minutes in the future. Verify the source system clock.</P> <H3>&nbsp;</H3> <H3>"A processing rule doesn't seem to be working"</H3> <P>Query for <CODE>processing_transformer_error</CODE>, <CODE>processing_transformer_timeout</CODE>, or <CODE>processing_batch_timeout</CODE>. → If present: the rule encountered an error or timeout. Review the rule configuration.</P> <H3>&nbsp;</H3> <H3>"Records arrived but <CODE>dt.ingest.warnings</CODE> is absent"</H3> <P>The record was processed without any limit being applied. The issue is not an ingestion limit&nbsp; investigate the configuration (ingest rule, OpenPipeline routing, bucket assignment) instead.</P> <P>&nbsp;</P> <H2>What's next</H2> <P><EM><STRONG>Opening a support case&nbsp; with below details</STRONG></EM><EM><BR />&nbsp; &nbsp; &nbsp; &nbsp;</EM></P> <UL> <LI>DQL query result showing affected <CODE>dt.ingest.warnings</CODE> values and counts</LI> <LI>Sanitized sample record showing the full <CODE>dt.ingest.warnings</CODE> array and the affected fields</LI> <LI>OpenPipeline processing rule configuration (sanitized), if <CODE>_pipe</CODE> warnings are present</LI> <LI>Source system and ingestion method (OneAgent, API, OpenTelemetry, Fluent Bit, etc.)</LI> <LI>Applicable OneAgent, ActiveGate, or Operator version</LI> </UL> <H2><BR />Related reading</H2> <UL> <LI><A href="https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-troubleshooting/lma-ingest-warnings" target="_blank" rel="noopener noreferrer">Log ingestion warnings&nbsp; official reference</A></LI> <LI><A href="https://docs.dynatrace.com/docs/analyze-explore-automate/log-monitoring/lmc-troubleshooting/lmc-ingest-warnings" target="_blank" rel="noopener noreferrer">Log ingestion warnings&nbsp; Log Monitoring Classic</A></LI> <LI><A href="https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-limits" target="_blank" rel="noopener noreferrer">Log ingestion limits</A></LI> <LI><A href="https://community.dynatrace.com/t5/tkb/workflowpage/tkb-id/troubleshooting/article-id/1188" target="_blank" rel="noopener noreferrer">My logs went silent where do I start?</A></LI> <LI><A href="https://community.dynatrace.com/t5/Troubleshooting/Troubleshooting-JSON-parsing-in-Log-module/ta-p/300189" target="_blank" rel="noopener noreferrer">Troubleshooting JSON Parsing in Log Module</A></LI> <LI><A href="https://www.dynatrace.com/news/blog/modern-log-management-resilience-and-self-service-upgrades-for-log-ingestion/" target="_blank" rel="noopener noreferrer">Improve log ingest health with smarter log management</A></LI> <LI><A href="https://community.dynatrace.com/t5/Troubleshooting/Logs-Troubleshooting-Map/ta-p/302260" target="_blank" rel="noopener noreferrer">Dynatrace Logs Troubleshooting Map</A></LI> </UL> </DIV> Fri, 04 Sep 2026 06:49:17 GMT noel_david 2026-09-04T06:49:17Z https://community.dynatrace.com/t5/Troubleshooting/What-do-log-ingest-warnings-on-my-records-mean-dt-ingest/ta-p/303795 <P><LI-TOC indent="15" liststyle="disc" maxheadinglevel="2"></LI-TOC></P> <DIV class="lia-message-template-content-zone"> <P>*<EM>Use a table of contents for longer articles.&nbsp;</EM></P> <H2>Summary</H2> <P><EM>*This is a mandatory section.</EM></P> <P><EM>Short description of which part of the Dynatrace platform the article refers to and what kind of problem it will help resolve/ task it will describe.</EM></P> <P>&nbsp;</P> <H2>Problem</H2> <P><EM>*This is a mandatory section.</EM></P> <P><EM>Provide a precise description of the problem/ task to be described. Use anonymized screenshots, and include text for&nbsp;important messages, errors, or information that will help the customer find this article when searching.</EM></P> <P>&nbsp;</P> <H2>Troubleshooting steps</H2> <P><EM>*This section can be omitted as necessary.</EM></P> <P><EM>Explain what troubleshooting steps should be taken to ensure the problem matches this article.</EM></P> <P>&nbsp;</P> <H2>Resolution</H2> <P><EM>*This section can be omitted for articles that guide customers on case creation.</EM></P> <P><EM>Explain the solution or all possible solutions resulting from the troubleshooting steps.<BR /><BR /></EM></P> <H2>What's next</H2> <P><EM>*This is a mandatory section. Customers need a way to respond or follow up if they have questions.&nbsp;</EM></P> <P><EM>Explain what to do if the article didn't help.&nbsp;</EM><EM>Note that there are multiple options available, including:</EM></P> <OL> <LI><EM><STRONG>Opening a chat or support case&nbsp;</STRONG>- be as specific as possible about the information the customer should include in the case.&nbsp;</EM><EM>Use the following formula and fill or change the bullet points:<BR /><BR /></EM> If this article didn't help, <STRONG>create a chat or open a support case</STRONG> <UL> <LI>Mention that you reviewed this article.</LI> <LI>Include the following details: <UL> <LI>Link to XYZ</LI> <LI>Screenshot of XYZ</LI> <LI>Information about XYZ</LI> </UL> </LI> </UL> </LI> <LI><EM><STRONG>Suggesting Product Idea</STRONG> - encourage the customer to suggest/ vote for a Product Idea explaining their business use case.<BR /></EM> <DIV>Please submit a Product Idea with detailed information about your request and use case in the Product Ideas forum. Our Product Management team regularly reviews these suggestions:</DIV> <A href="https://community.dynatrace.com/t5/Dynatrace-product-ideas/idb-p/DynatraceProductIdeas" target="_blank" rel="noopener">https://community.dynatrace.com/t5/Dynatrace-product-ideas/idb-p/DynatraceProductIdeas</A></LI> <LI><STRONG><EM>Explain this will change in the future </EM></STRONG><EM>- explain that this behavior will change in a future release. (No product idea / support case needed)</EM></LI> <LI><STRONG><EM>Alternatives -</EM></STRONG><EM>&nbsp;any other actions or links to other articles that could move the customer forward.</EM></LI> </OL> <H2><BR />Related reading</H2> <P><EM>*Link this article to other relevant configuration or troubleshooting guides, including any applicable troubleshooting maps.<BR />:open_book:&nbsp;&nbsp;</EM><A href="https://community.dynatrace.com/t5/Troubleshooting/" target="_blank" rel="noopener">XXX Troubleshooting Map</A></P> <P>:open_book:&nbsp; Link 1</P> <P>:open_book:&nbsp; Link 2</P> <P>:open_book:&nbsp; Link 3</P> <P>:open_book:&nbsp; Link 4</P> </DIV> Fri, 04 Sep 2026 06:49:17 GMT https://community.dynatrace.com/t5/Troubleshooting/What-do-log-ingest-warnings-on-my-records-mean-dt-ingest/ta-p/303795 noel_david 2026-09-04T06:49:17Z