03 Dec 2020 04:50 PM
Hello I have a Openshift4 cluster with CRI-O containerezation. Trying to instument some java application with dynatrace-operator. I see - dynatrace-agent successfuly deploy and connected to ActiveGate, but in logs I see error:
2020-12-03 13:50:15.845 UTC [43ff880d] info [native] Generating group id data for process group: 10232776891962716058, group name: SpringBoot App *********-* took: 4212ms. (mmap read time: 0 ms)
2020-12-03 13:50:15.925 UTC [43ff880d] warning [native] Update metrics took: 5809 ms, collecting host metrics: 1 ms, collecting disk metrics: 123 ms, collecting network metrics: 106 ms, detecting processes: 1281 ms, reporting os runtime configuration: 0 ms, updating network agent metrics correlator : 0 ms, filtering docker bridges: 0 ms, host events generation: 1 ms, grouping processes: 4218 ms, handling blacklist: 0 ms, group events generation: 1 ms, updating aggregator: 4 ms, resource exhausted events: 1 ms, updating host info: 73 ms, deployment changes storing: 0 ms, logging monitored processes: 0 ms, closing child processes: 0
2020-12-03 13:50:15.925 UTC [43ff880d] warning [native] Timer task 'updateMetrics' took 5809ms to execute (which is more than half of the timer's period)
2020-12-03 13:50:15.926 UTC [43ff880d] info [native] Detected PA Injection failure for PGI:10232776891962716058 (0x8e02205e1f436f9a), name:/tmp/*********.jar *********-* *********, process data: Process PID: 3524072
Process parent PID: 3524015
Process executable: java
Process path: /usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/bin/java
Working directory: /
Process name: java
Argv zero: java
Command line parameters count: 15
-Dfile.encoding=UTF-8
-XX:+UseContainerSupport
-XX:InitialRAMPercentage=30
-XX:MaxRAMPercentage=75
-XX:MaxMetaspaceSize=256M
-XX:ActiveProcessorCount=1
-XX:MaxRAM=1000m
-XX:+UnlockExperimentalVMOptions
-XX:InitialRAMPercentage=50
-XX:MaxRAMPercentage=70
-XX:G1HeapRegionSize=2M
-XX:G1NewSizePercent=30
-XX:G1MaxNewSizePercent=70
-jar
/tmp/*********.jar
Physical path: /proc/3524072/root/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/bin/java
Physical working directory: /proc/3524072/root/
Working set size: 354381824
Virtual memory peak size: 3874779136
Previous virtual memory peak size: 3874779136
CPU utilization: 6.28976
last CPU val: 173
Process virtual machine: Java
Java mainClass: /tmp/*********.jar
Process loaded libraries (22) : /tmp/hsperfdata_1000920000/15|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/bin/java|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/jli/libjli.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/libjava.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/libjimage.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/libnet.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/libnio.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/libverify.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/libzip.so|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/modules|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/server/classes.jsa|/usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/lib/server/libjvm.so|/usr/lib64/ld-2.17.so|/usr/lib64/libc-2.17.so|/usr/lib64/libdl-2.17.so|/usr/lib64/libgcc_s-4.8.5-20150702.so.1|/usr/lib64/libm-2.17.so|/usr/lib64/libnss_files-2.17.so|/usr/lib64/libpthread-2.17.so|/usr/lib64/librt-2.17.so|/usr/lib64/libstdc++.so.6.0.19|/usr/lib64/libz.so.1.2.7
Observed measurement cycles: 1
Group id params: 7
JAVA_JAR_FILE | *********.jar
SPRINGBOOT_STARTUP_CLASS | App
EXE_NAME | java
KUBERNETES_CONTAINERNAME | *********
KUBERNETES_BASEPODNAME | *********-*
KUBERNETES_NAMESPACE | *********
HOST_GROUP | OpenShift4
Node id params: 5
JAVA_JAR_PATH | /tmp/*********.jar
EXE_PATH | /usr/lib/jvm/java-*-openjdk-*-*.el*_*.x*_*/bin/java
KUBERNETES_FULLPODNAME | *********-213-5jxrv
KUBERNETES_PODUID | 14135a1b-5b89-486e-8294-a0b357d5492b
COMMAND_LINE_ARGS | /usr/lib/jvm/java-11-openjdk-11.0.3.7-0.el7_6.x86_64/bin/java -Dfile.encoding=UTF-8 -XX:+UseContainerSupport -XX:InitialRAMPercentage=30 -XX:MaxRAMPercentage=75 -XX:MaxMetaspaceSize=256M -XX:ActiveProcessorCount=1 -XX:MaxRAM=1000m -XX:+UnlockExperimentalVMOptions -XX:InitialRAMPercentage=50 -XX:MaxRAMPercentage=70 -XX:G1HeapRegionSize=2M -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=70 -jar /tmp/*********.jar
Container Data:
Container ID: 0a6cf3a912ea48be485d35ef78936d76c34ec25f00ef87434805282646d55cd9
PID: 3524015
Created: 2020-12-03T13:49:48.900331312Z
Image ID: registry.localhost.ru/dev/*********/*********/*********@sha256:b025943b0f5faed7eb5e2677c18beff2051c2824df11c2f8ca4415d986aeb7d0
Image name: registry.localhost.ru/dev/*********/*********/*********@sha256:b025943b0f5faed7eb5e2677c18beff2051c2824df11c2f8ca4415d986aeb7d0
Orchestration layer data: KubernetesData:
Container name: *********
Base POD name: *********-213
POD name: *********-213-5jxrv
POD UID: 14135a1b-5b89-486e-8294-a0b357d5492b
K8s namespace: *********
K8s pod Ip address: 10.128.125.14
HI Level Container Runtime: crio
, failure reason:Container injection failed
2020-12-03 13:50:20.114 UTC [43ff880d] info [native] Received Throttler non active: 0
2020-12-03 13:50:21.636 UTC [43ff880d] info [native] Update metrics took: 1522 ms, collecting host metrics: 1 ms, collecting disk metrics: 122 ms, collecting network metrics: 102 ms, detecting processes: 1284 ms, reporting os runtime configuration: 1 ms, updating network agent metrics correlator : 0 ms, filtering docker bridges: 0 ms, host events generation: 0 ms, grouping processes: 4 ms, handling blacklist: 0 ms, group events generation: 1 ms, updating aggregator: 4 ms, resource exhausted events: 0 ms, updating host info: 3 ms, deployment changes storing: 0 ms, logging monitored processes: 0 ms, closing child processes: 0
How to troubleshoot this problem?
Thanks
31 Dec 2020 08:39 PM
30 May 2025 10:45 AM
Hey @sytar_alex , did you manage to find the solution to your problem? If so, it would be amazing if you've shared it with the rest of the Community! If not, let me know, and I'll look for some further assistance 😊