Logs Not Being Ingested Due to F5 Distributed Cloud WAAP Intercepting OneAgent Communication

[nikitachauhan]

Summary

This article applies to Dynatrace OneAgent communication and Log Monitoring. It helps troubleshoot situations where logs are delayed, intermittently ingested, or completely missing because OneAgent communication is being intercepted by an F5 Distributed Cloud WAAP (formerly Volterra) deployment positioned in front of a Dynatrace Cluster.

The article explains how to identify the issue using OneAgent logs and communication traces and how to work with network teams to restore normal log ingestion.

---

Problem

Customers may observe one or more of the following symptoms:

• Logs stop appearing in Dynatrace.
• Log ingestion becomes significantly delayed.
• Some hosts continue sending logs while others stop completely.
• OneAgent appears healthy.
• No Dynatrace-side ingestion errors are visible.
• File uploads or communication requests fail intermittently.

OneAgent logs may contain messages similar to:

Did not expect to receive data on asynchronous channel - ignoring

or

Could not parse response from server

In affected environments, communication requests may appear successful because the HTTP request returns a valid status code:

HTTP/1.1 200 OK

Despite the successful HTTP response, the response body is generated by an intermediary security device instead of the Dynatrace Cluster.

---

Troubleshooting Steps

Step 1: Review OneAgent Communication Logs

Search the OneAgent logs for:

Did not expect to receive data on asynchronous channel - ignoring

This message indicates that OneAgent received a response body when it expected no response body.Responses containing data are unexpected and typically indicate traffic interception or modification.

---

Step 2: Check Communication Traces

Review the communication log immediately before the error.

Example:

POST /communication HTTP/1.1
Host: <cluster-url>

HTTP/1.1 200 OK
server: volt-adc
content-length: 269
x-volterra-location: mb2-mum

Did not expect to receive data on asynchronous channel - ignoring

The following response headers strongly indicate that traffic is traversing F5 Distributed Cloud infrastructure:

• server: volt-adc
• x-volterra-location

These headers are not generated by Dynatrace.

---

Step 3: Look for Fixed-Length Responses

Another key indicator is the presence of a repeated response length:

content-length: 269

During investigation, request sizes varied significantly while response sizes remained consistently 269 bytes.

This behavior strongly suggests a proxy-generated response rather than a Dynatrace protocol response.

---

Step 4: Search for Parsing Failures

Search OneAgent logs for:

Could not parse response from server

Example:

FileUploadStream: Upload of zipped stream failed

Could not parse response from server:
269 bytes beginning with ...

This indicates that OneAgent expected a Dynatrace protocol response but instead received HTML content.

Receiving HTML in response to Dynatrace cluster communication is a strong indicator that another system generated the response before it reached the Dynatrace backend.

---

Step 5: Verify Reverse Proxy or WAAP Deployment

Work with the network or security team to determine whether any of the following components exist between OneAgent and the Dynatrace Cluster:

• F5 Distributed Cloud WAAP
• Volterra
• Web Application Firewall (WAF)
• Application Delivery Controller (ADC)
• DDoS protection platform
• API gateway or API protection solution
• Bot defense platform

Particular attention should be given to products providing:

• DDoS protection
• Rate limiting
• Intrusion prevention
• Bot defense
• Request filtering
• Content inspection
• Traffic rewriting

---

Resolution

The root cause may be that an F5 Distributed Cloud WAAP deployment is intercepting OneAgent communication before it reaches the Dynatrace Cluster.

During investigation, the following observations were made:

• OneAgent successfully established communication sessions.
• HTTP requests returned 200 OK.
• Responses contained Volterra-specific headers.
• Response bodies consistently measured 269 bytes.
• OneAgent received HTML instead of Dynatrace protocol responses.
• Communication uploads failed with parsing errors.

OneAgent expects protocol-specific binary responses from Dynatrace. When a security device generates its own response instead, OneAgent cannot process the data and communication failures occur.

To resolve the issue:

1. Review F5 Distributed Cloud WAAP policies.
2. Allow Dynatrace OneAgent traffic to pass through without modification.
3. Exclude Dynatrace communication endpoints from inspection when appropriate.
4. Review rate-limiting, bot-defense, WAF, and API security controls.
5. Verify that requests reach the Dynatrace Cluster directly.
6. Confirm that communication errors disappear after the policy change.

After appropriate exclusions or allowlisting are implemented, log ingestion should return to normal.

---

What's Next

Opening a Support Ticket

If this article did not resolve the issue, please open a Dynatrace Support ticket and mention that you followed this article.

Include the following information:

• OneAgent support archive.
• Affected host names.
• OneAgent version.
• Dynatrace Cluster URL.
• Screenshots showing delayed or missing logs.
• Details of any reverse proxy, WAAP, WAF, ADC, API gateway, or load balancer positioned between OneAgent and the Dynatrace Cluster.

---

Alternatives

If a reverse proxy is required in your environment:

• Work with the security team to allowlist Dynatrace OneAgent communication.
• Review WAAP logs for dropped, rejected, or modified requests.
• Compare traffic from affected and unaffected hosts.
• Validate that the /communication endpoint is not inspected or modified.

---

More Information

More troubleshooting articles can be found on the Logs Troubleshooting Map – Community Home.