Basic question : I am wondering if there is a way to have dynatrace use the AWS trace header x-amzn-trace-id as the traceid in distributed traces.
We have AWS setup using a ALB and api gateway. The gateway calls an authorizer lambda and then calls the api endpoint lambda.
AWS will generate a x-amzn-trace-id that correlates to these two lambda calls. However dynatrace shows each of these as separate Trace Ids in the distributed traces.
If I supply a traceparent header to my request, then I see both requests in the trace having the same traceid.
We cannot simply create the w3c tracing headers as we don't own the calling application, so we would need to inject them somehow or make dynatace use the standard AWS trace headers.
My preference being using the AWS trace headers.
If there is not a way is there a document that describes how the oneagent processes the headers supplied to create the trace id in dynatrace? I assume there is some sort of order of precendece of x-dynatrace traceparent, etc. Maybe there is a clue there on how I could map the amz trace headers.
there are many pitfalls in handling multiple trace-context standards. Today Dynatrace supports it's own tracecontext + W3C TraceContext used e.g. in OpenTelemetry which is going to be adopted by more and more tracing systems. We do not plan to natively support X-Ray's tracecontext within our agent as also AWS is improving interoperability with OpenTelemetry. As a first step they just announced e.g. AWS X-Ray now supports W3C format trace IDs for distributed tracing (amazon.com) to be able using W3C TraceContext for ingesting trace-data into X-Ray. Yet this doesn't solve the requirement of aws services to natively handle W3C TraceContext.
For the time being, the proposed solution is to capture x-amzn-trace-id (and other correlation id's such as x-amz-request-id, x-correlation-id) via request attributes to be able to filter/follow causally related traces.
We do plan to automatically capture these additional id's for correlation as part of our enhanced capabilities around the concept of span-links and custom trace correlation using the power of DQL to show and analyze end-2-end traces that share one or many attributes.