cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

The OneAgent layer, in a subset of AWS lambda functions in Java, does not send monitoring data

stripathi
Visitor

We have a set of AWS lambda function written in Java which are not able to post data to & do not appear in Dynatrace. However at the same time many others are able to appear on & post data to Dynatrace.

All the function are running on memory recommended which 1.5 GB (1500 MB). For brevity, I am keeping this post limited to one function which has even additional memory: 3000 MB (3 GB).

I compared cloudwatch logs of the both the function, passing & failing the Dynatrace communication. The Passing function has the logs below. 

The logs of failing function have the <start-section> but  do not have the <End section> of logs which executes flush and send operation. I think the process of lambda function shuts down before OneAgent can write the data to Dynatrace.

Please assists to identify the potential factors responsible, solutions or steps for further investigation on this issue.

 

<Function invoked: start section>
[Dynatrace] <date..> UTC [00000001] [Logger] ========= Begin logging of 8 buffered messages =========
[Dynatrace] <date..> UTC [00000001] [Global] Expanded "log-stdout"="true" from "logging/destination"="stdout"
[Dynatrace] <date..> UTC [00000001] [Global] Expanded "log-Transformer"="true" from "logging/java/flags"="log-Transformer=true,log-EmbeddedJarClassLoader=true,log-OpenTelemetryUtils=true,log-AsyncClassRetransformer=true,log-ClassValue=true"
[Dynatrace] <date..> UTC [00000001] [Global] Expanded "log-EmbeddedJarClassLoader"="true" from "logging/java/flags"="log-Transformer=true,log-EmbeddedJarClassLoader=true,log-OpenTelemetryUtils=true,log-AsyncClassRetransformer=true,log-ClassValue=true"
...
<mid section omitted for brevity>
...
<End section>
[Dynatrace] <date...> [00000001] [OpenTelemetryUtils] Force flushing spans ...
[Dynatrace] <date...> [0000000b] [OpenTelemetryUtils] Starting last chance resource callback ...
[Dynatrace] <date...> [0000000b] [OpenTelemetryUtils] + Finished last chance resource callback, adding: Resource{schemaUrl=null, attributes={}}
[Dynatrace] <date...> [0000000b] [Transformer] Opened 'module java.base' package 'sun.net.www.http' to blackbox module.
[Dynatrace] <date...> [0000000b] [Transformer] Opened 'module java.base' package 'sun.security.ssl' to blackbox module.
[Dynatrace] <date...> [0000000b] [com.dynatrace.opentelemetry.DynatraceHttpConnection] WARNING span upload: HTTPS://{environmentid}.live.dynatrace.com
        DT_CONNECTION_AUTH_TOKEN: !Ref DtConnectionAuthToken
        DT_LOGGING_DESTINATION: stdout
        DT_LOGGING_JAVA_FLAGS: log-Transformer=true,log-EmbeddedJarClassLoader=true,log-OpenTelemetryUtils=true,log-AsyncClassRetransformer=true,log-ClassValue=true
        DT_ENABLE_LAMBDA_EXTENSION_REGISTRATION: false
        DT_OPEN_TELEMETRY_ENABLE_INTEGRATION: true

 

 


AWS details:

Memory: 3072MB

Ephemeral storage: 512MB

Edit:

 I noticed formatting issue after the edit @aciejNeumann. The last section starting from {environment}.live.dyantrace.com is not the part of the actual log but is the set of environment variables supplied to lambda. However I am not able to fix it because the code section won't let me edit itself

1 REPLY 1

ChadTurner
DynaMight Legend
DynaMight Legend

@stripathi are you still having this issue? If so, I recommend tossing in a support ticket to have this issue addressed.

-Chad

Featured Posts