on - edited on by
Michal_Gebacki
- The problem
- Verify that the issue is a RUM correlation issue
- Distributed trace was not captured
- User action was not captured
- Verify that RUM correlation is supported in your scenario
- Verify that CORS handling is set up correctly
- Identify common issues
- Disabled OneAgent functionality
- Infrastructure stripping request headers and cookies
- Infrastructure adding the HttpOnly attribute to the dtCookie
- Cookie domain issues
- Long-running requests
- FAQ
- Is this guide also applicable in manual insertion scenarios?
- Why is theView tracebutton not displayed even though the trace is available in the drilldown above t...
- What's Next?
The problem
Dynatrace RUM correlates user actions with their corresponding distributed traces to provide seamless end-to-end visibility. This correlation enables features like:
- Drill-down from the waterfall analysis chart to distributed traces
- Linking services to RUM applications
- Creation of user tags and user action and session properties from server-side request properties
RUM correlation issues occur when the trace captured for a web request is not linked to its corresponding user action. This guide helps you systematically analyze such issues. We recommend working through the steps in sequence for best results.
The guide includes screenshots from Telerik Fiddler, but you can also use your browser’s developer tools to follow along.
Verify that the issue is a RUM correlation issue
To diagnose any RUM correlation issue, you first need to identify both the user action and the distributed trace that you expect to be correlated. Use the Distributed Traces screen to locate the trace you expect to be correlated—narrow the timeframe and apply filtering options to refine your search.
Doing so helps you distinguish RUM correlation problems from other types of issues, such as the following:
Distributed trace was not captured
A missing drilldown to a distributed trace for a specific request in the waterfall analysis of a user action is not, by itself, sufficient evidence of a correlation issue. It’s possible that no distributed trace was captured in the first place.
Common root causes for missing traces include:
- Adaptive traffic management is active. If a trace wasn’t captured due to adaptive traffic management, a message indicating this will appear instead of the View trace button when you hover over the resource in the waterfall analysis.
- The resource was served from a cache rather than the instrumented server—for example, via a CDN.
- If the trace is consistently missing, verify that the process group is properly instrumented, full-stack monitoring is enabled, and trace capturing isn’t suppressed by a URL exclusion rule (Settings → Server-side service monitoring → Deep monitoring → Exclude specific incoming web request URLs).
User action was not captured
Conversely, the presence of a distributed trace that isn’t correlated to a user action is not, by itself, sufficient evidence of a RUM correlation issue. It’s possible that no user action was captured in the first place.
Sometimes, the Calls from unmonitored hosts, pages or devices shown in a service’s backtrace are misinterpreted as signs of RUM correlation problems. A web request appears here if it isn’t linked to a RUM user action—so while correlation issues can contribute, so can cases where no user action was captured at all. For this reason, the backtrace is not a reliable tool for diagnosing correlation problems.
There are many possible reasons why a user action might not be captured, including:
- Script or ad blockers installed in end users' browsers
- Requests not originating from a browser
- Opt-in mode
- Cost and traffic control
- Overload prevention
- Browser exclusions
- IP exclusions
- RUM is disabled for "Do Not Track"-enabled browsers
- Use of a technology that does not support automatic injection and therefore requires agentless monitoring
Verify that RUM correlation is supported in your scenario
Out-of-the-box RUM correlation is not supported in all scenarios—and in some cases, it’s not supported at all. The key factors to consider are:
- The same-origin policy, a browser-enforced security mechanism.
- The technology used on the first tier instrumented with OneAgent. The list of technologies with full or limited RUM correlation support is available here: Technology support - Real User Monitoring - Web servers and applications. For AWS Lambda, correlation can be set up for XHRs. For other technologies, RUM correlation is currently not supported.
These factors can vary across different requests within the same application. Therefore, if you encounter RUM correlation issues, it’s important to examine individual requests. Use the decision tree below to determine what applies to the specific web request you're analyzing. For definitions of "same-origin" and "same-site", refer to Understanding "same-site" and "same-origin".
In scenarios where the decision tree calls for the use of the x-dtc header or for CORS requests with credentials, additional configuration is required. See Link cross-origin XHR user actions and their distributed traces for setup instructions.
Technology-Specific Configuration
For the following technologies, you need to enable specific OneAgent features:
- Kestrel: Enable Real User Monitoring (RUM) for ASP.NET Core and .NET ASP.NET Sensor V2.
- Netty: Enable Netty Real User Monitoring (RUM) to distributed trace correlation. For environments created with version 1.293 or later, this feature is enabled by default. Depending on your OneAgent version, you may also need to activate the toggle Activate this feature also in OneAgents only fulfilling the minimum Opt-in version.
You can enable OneAgent features via Settings → Preferences → OneAgent features or Your process group → Settings → OneAgent features. See OneAgent features for details.
For other supported technologies, the required OneAgent features for RUM correlation are enabled by default. These include:
- Real User Monitoring
- Web server
- Java servlet
- Node.js web request
For Java, note that the process must be restarted after enabling a feature for the changes to take effect.
Considerations when using OpenTelemetry
There are two ways to integrate OpenTelemetry with Dynatrace:
- Using the OneAgent span sensor – see OpenTelemetry traces with OneAgent
- Using OpenTelemetry trace ingest
If the trace is enriched using the OneAgent span sensor, you can apply the decision tree above as-is.
If OpenTelemetry trace ingest is used and no part of the trace is captured using OneAgent, RUM correlation is not possible. If part of the trace is captured using OpenTelemetry and another part by OneAgent, identify the segment captured by OneAgent and apply the decision tree to that segment.
Workaround for Envoy
The OneAgent code module for Envoy does not support RUM. To achieve RUM correlation in scenarios where a RUM-capable OneAgent is deployed behind Envoy, you must disable OneAgent’s Envoy monitoring. Starting with Envoy version 1.29, a replacement integration is available that leverages OpenTelemetry and does not interfere with RUM correlation. For setup instructions, see Configure Envoy for Dynatrace.
Verify that CORS handling is set up correctly
In cross-origin scenarios, the decision tree requires the use of the x-dtc header or CORS requests with credentials to propagate the information necessary for RUM correlation. The following sections guide you through verifying that your setup meets these requirements.
x-dtc header
To allow the RUM JavaScript to add the x-dtc header to requests, you need to define a regular expression that matches the target URL in the application configuration. For details, see Link cross-origin XHR user actions and their distributed traces.
Adding the header triggers a browser preflight request—an OPTIONS request—which must be properly handled by your endpoints. If not, a CORS error will appear in the browser console.
Specifically, the preflight response must include the Access-Control-Allow-Headers: x-dtc header to permit the use of the x-dtc header. It must also include an Access-Control-Allow-Origin header that allows the requesting origin. Additionally, depending on the HTTP method used, the response may need to include an Access-Control-Allow-Methods header.
If the preflight request succeeds, the browser includes the x-dtc header in the actual request:
Note that only the actual request can be correlated—not the preflight request.
CORS requests with credentials
A CORS request with credentials is triggered when the monitored application:
- Uses the
withCredentialsproperty with XMLHttpRequest - Sets the credentials mode to
includewith the Fetch API - Adds the
crossorigin="use-credentials"attribute to HTML elements
The server must respond with an Access-Control-Allow-Credentials: true header and a specific Access-Control-Allow-Origin header (that is, not a wildcard). Failure to include these headers will result in a CORS error in the browser console.
You can confirm this setup by checking whether the browser sends RUM cookies, such as the dtCookie, on the request.
RUM cookies are only sent if they are applicable. Typically, you don’t need to explicitly configure the cookie domain, as both the RUM JavaScript and OneAgent automatically select the broadest possible domain. However, if cookies are missing from the request, verify that the cookie domain configuration does not restrict them to a subdomain that excludes the request in question.
Identify common issues
If RUM correlation is supported according to the decision tree and, where necessary, CORS handling is set up correctly, but the user action and its corresponding trace still aren't linked, review the following common issues.
Disabled OneAgent functionality
To ensure that the OneAgent functionality required for RUM correlation is active, follow these steps:
- Confirm that the OneAgent features listed in the Technology-Specific Configuration section are enabled.
- Make sure that Real User Monitoring is enabled for the process group on the first instrumented tier: Your process group → Settings → Real User Monitoring → Enable Real User Monitoring
For Java, note that the process must be restarted after enabling the feature for the changes to take effect.
In case you disabled the process group setting Enable Real User Monitoring to prevent automatic injection and opted to insert the RUM JavaScript manually, use a custom injection rule instead, as described in Use manual insertion for pages of an auto-injected application .
Infrastructure stripping request headers and cookies
For RUM correlation to work reliably, it’s essential that the firewall constraints for RUM are respected and that RUM-related headers and cookies are properly propagated between the RUM JavaScript and all involved RUM-capable OneAgents.
One key cookie used for RUM correlation is the dtCookie, which OneAgent can create and update. If the dtCookie is stripped from the Cookie request header before reaching the OneAgent, the OneAgent will generate a new one and return it in the Set-Cookie response header.
This newly generated dtCookie will have a different session ID and often a different server ID than the original as shown in the screenshot below. These mismatched identifiers prevent successful RUM correlation.
Infrastructure adding the HttpOnly attribute to the dtCookie
The HttpOnly attribute is not supported for RUM cookies, as it prevents the RUM JavaScript from accessing them. If your infrastructure adds it to RUM cookies, this will lead to RUM correlation issues—among other problems. To resolve this, the attribute must be removed.
How to Check for HttpOnly
To determine whether your infrastructure is adding the HttpOnly attribute, open the Application tab in Chrome DevTools and check whether the dtCookie is marked as HttpOnly.
In the Network tab, inspect the document request and examine the Set-Cookie response header that sets the dtCookie. If the header includes the HttpOnly attribute, it confirms the issue.
Cookie domain issues
If the domain assigned to the dtCookie does not match the domain of the request—as shown in the screenshot below—RUM correlation will fail.
This mismatch can occur for two reasons:
- Automatic domain detection fails because host name determination was not configured correctly.
- The cookie domain was manually configured to a domain that doesn't cover the request domain.
For more details, refer to Configure the RUM cookie domain for web applications.
Long-running requests
Check the response time of the uncorrelated trace by clicking on its root node—the topmost node in the trace hierarchy—and opening the Timing tab. If the response time exceeds 3 minutes, reliable RUM correlation is no longer possible.
FAQ
Is this guide also applicable in manual insertion scenarios?
Yes. The ability to correlate traces with user actions does not depend on how the RUM JavaScript is added to the HTML—whether it’s automatically injected or manually inserted—nor on the application type (auto-injected or manually inserted). Therefore, this guide is fully applicable in manual insertion scenarios.
Why is the View trace button not displayed even though the trace is available in the drilldown above the waterfall?
In some cases, the View trace button is not shown when hovering over a resource in the waterfall analysis, even though the trace is available via the drilldown above the chart.
In such cases, the trace was successfully correlated with the user action, but the matching resource timing could not be identified. This typically occurs when the URL was rewritten and the required Server-Timing response header—used to identify the resource timing—is missing.
This can happen in the following scenarios:
- The technology on the first instrumented tier is AWS Lambda, Netty, MuleSoft HTTP Listener, WebMethods Integration Server, Undertow, or Apache HttpCore. For these technologies, the addition of the
Server-Timingheader is not supported. - The header was stripped by a component in your infrastructure. In this case, make sure your components are configured according to the firewall constraints for RUM.
What's Next?
If, after completing the troubleshooting steps above, the trace is still not correlated with its user action, please contact Dynatrace via the in-product chat or by creating a support ticket.
To help our support team investigate efficiently, include the following information:
- A link to the affected user action.
- A link to the distributed trace you expect to be correlated.
- A HAR file capturing the full scenario in which the request is sent. See How to generate a HAR file.
- A brief summary of the troubleshooting steps you've already completed.
Providing this information upfront will help accelerate the resolution process.
