topic NGINX logs enrichment whether Dynatrace is enabled or not in Log Analytics

NGINX logs enrichment whether Dynatrace is enabled or not

ahmad — Tue, 13 Jun 2023 11:25:43 GMT

Hello all,

I've been following the article to enable Nginx logs enrichment and to include the DT in the Nginx logs however, I have Dynatrace implemented in one environment and it works fine but when I try to run Nginx on my local machine I get the following error:

[emerg] 1#1: unknown "dt_trace_id" variable nginx: [emerg] unknown "dt_trace_id" variable

My config looks like this:

worker_processes auto; error_log /var/log/nginx/error.log notice; pid /tmp/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format custom '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '[!dt dt.trace_id=$dt_trace_id,dt.span_id=$dt_span_id,dt.trace_sampled=$dt_trace_sampled] ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log custom; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; include /etc/nginx/conf.d/*.conf; }

Anyone can advise on how I can do Nginx logs enrichment that can work whether Dynatrace is enabled or not?

Thank you.

Re: NGINX logs enrichment whether Dynatrace is enabled or not

ChadTurner — Thu, 03 Aug 2023 19:41:57 GMT

@ahmad were you able to solve this issue or are you still experiencing it? If it was solved, what was the underlying problem?

Re: NGINX logs enrichment whether Dynatrace is enabled or not

danmichael — Thu, 14 Dec 2023 14:44:18 GMT

Is nginx instrumented locally? What version of nginx is being run? Do you have the oneagent installed?

Nginx logs enrichment that can work whether Dynatrace is enabled or not? -- using this method depends on the nginx process being instrumented by the OneAgent

Re: NGINX logs enrichment whether Dynatrace is enabled or not

jasonbutler22 — Wed, 25 Jun 2025 00:07:35 GMT

We also have dev and test environments without Dynatrace. My strategy is to run a startup script to detect if Dynatrace is instrumenting the Docker container my nginx server is running in. The script tests if the /etc/ld.so.preload is set to load the dynatrace one agent library. If it is detected then I update the logging config with the Dynatrace specific logging patterns. e.g.

# Check if the Dynatrace OneAgent is installed by checking for the presence of
# liboneagentproc in the /etc/ld.so.preload file.
if grep -q "liboneagentproc" "/etc/ld.so.preload" ; then 
echo "Enabling Dynatrace logging instrumentation"
# Update or replace nginx.conf to insert Dynatrace pattern [!dt dt.trace_id=$dt_trace_id]
# into the logging config here.
fi

Re: NGINX logs enrichment whether Dynatrace is enabled or not

radek_jasinski — Wed, 25 Jun 2025 07:49:36 GMT

Hi,

You can try solving this by defining fallback values for the Dynatrace variables using the map directive in NGINX. This is useful when Dynatrace OneAgent is not installed (e.g., in local environments), and NGINX would otherwise throw an error due to unknown variables like $dt_trace_id.

map "" $dt_trace_id_safe {
default "-";
}

map "" $dt_span_id_safe {
default "-";
}

map "" $dt_trace_sampled_safe {
default "-";
}

then update your log_format.

Radek

Re: NGINX logs enrichment whether Dynatrace is enabled or not

andre_vdveen — Thu, 26 Jun 2025 13:12:39 GMT

Hello @radek_jasinski, do you have a working example of this that you can share? I'd like to see where you placed that and in which file, the nginx.conf or another? I also have a need for Nginx log enrichment, but I get the same error as @ahmad did.

I have the agent monitoring the nginx process in full stack, the OneAgent module dt_one_agent shows up in the modules list.

Re: NGINX logs enrichment whether Dynatrace is enabled or not

radek_jasinski — Fri, 27 Jun 2025 11:48:31 GMT

Yes, i will share later some example

Re: NGINX logs enrichment whether Dynatrace is enabled or not

Mo_Azuz — Tue, 19 Aug 2025 21:12:06 GMT

Hi @radek_jasinski, are you able to share any examples? Thank you!