Solved: GKE OneAgent Do Not Send Data My Dynatrace ActiveGate

salih_pala · ‎21 Feb 2020

Hello,

I have a problem and I want to ask you.

I installed OneAgent on Google kubernetes engine for to the information below.

https://www.dynatrace.com/support/help/technology-support/cloud-platforms/google-cloud-platform/google-kubernetes-engine/installation-and-operation/full-stack/deploy-oneagent-on-google-kubernetes-engine-clusters/

All pods seem to be working healthy. There is no error in his logs.

After this installation, I have reviewed the article below. I applied it later. When I add the kubernetes api and secret key on the Dynatrace homepage, I can see the cluster nodes. I see memory and cpu usage for my cluster nodes.. Its good.

https://www.dynatrace.com/support/help/technology-support/cloud-platforms/kubernetes/connect-kubernetes-clusters-to-dynatrace/

But I don't want to see this. I want to see my pods, services, communication with each other, apm data of my asp dotnet core application running in pods. however, no data on my pods appears on the Dynatrace. my pods, my services, nothing appears.

it only shows my node data in my cluster address which i added as api. it is already receiving node data via api, not oneagent. so the oneagent dynatrace doesn't seem to be sending data to my server.

.. Or data may be coming on Dynatrace. This may need to be activated by making a different setting.

Necessary information is given below. Please help me. where am i doing wrong

My gke cluster information;

Version: 1.13.12-gke.25

OS: Container-Optimized OS (cos)

Resources: Unlimited (autoscaling..)

------------------------------------

..Installation Steps..

------------------------------------

kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=$(gcloud config get-value account)

kubectl create namespace dynatrace

LATEST_RELEASE=$(curl -s https://api.github.com/repos/dynatrace/dynatrace-oneagent-operator/releases/latest | grep tag_name | cut -d '"' -f 4)

kubectl create -f https://raw.githubusercontent.com/Dynatrace/dynatrace-oneagent-operator/$LATEST_RELEASE/deploy/kubernetes.yaml

kubectl -n dynatrace logs -f deployment/dynatrace-oneagent-operator

kubectl -n dynatrace create secret generic oneagent --from-literal="apiToken=xxxxxxx" --from-literal="paasToken=xxxxxxx"

curl -o cr.yaml https://raw.githubusercontent.com/Dynatrace/dynatrace-oneagent-operator/$LATEST_RELEASE/deploy/cr.yaml

cat cr.yaml

apiVersion: dynatrace.com/v1alpha1

kind: OneAgent

metadata:

namespace: dynatrace

spec:

apiUrl: https://public-ip/e/xxx-xxx-xxx-xxx/api

skipCertCheck: true

disableAgentUpdate: true

tokens: "oneagent"

tolerations:

- effect: NoSchedule

key: node-role.kubernetes.io/master

operator: Exists

image: "docker.io/dynatrace/oneagent"

env:

- name: ONEAGENT_ENABLE_VOLUME_STORAGE

value: "true"

args:

- APP_LOG_CONTENT_ACCESS=1

------------------------------------------------

Results

------------------------------------------------

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

dynatrace-oneagent-operator-87cdf56bf-894kt 1/1 Running 0 17h 10.8.5.62 gke-xxx-project-cluster--xxx-project-cluster--d72271ba-99rl <none> <none>

oneagent-drq2r 1/1 Running 0 17h 10.26.1.56 gke-xxx-project-cluster--xxx-project-cluster--d72271ba-99rl <none> <none>

oneagent-f76g8 1/1 Running 0 17h 10.26.1.55 gke-xxx-project-cluster--xxx-project-cluster--3e09354d-js1m <none> <none>

oneagent-fb4ww 1/1 Running 0 17h 10.26.1.54 gke-xxx-project-cluster--xxx-project-cluster--3e09354d-0wwv <none> <none>

oneagent-fp8kp 1/1 Running 0 17h 10.26.1.53 gke-xxx-project-cluster--xxx-project-cluster--d72271ba-3jkg <none> <none>

oneagent-l2l8x 1/1 Running 0 17h 10.26.1.57 gke-xxx-project-cluster--xxx-project-cluster--89fb277a-gvp4 <none> <none>

oneagent-m2v4v 1/1 Running 0 17h 10.26.1.51 gke-xxx-project-cluster--xxx-project-cluster--d72271ba-1b5r <none> <none>

kubectl get deploy,daemonset

NAME READY UP-TO-DATE AVAILABLE AGE

deployment.extensions/dynatrace-oneagent-operator 1/1 1 1 18h

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE

daemonset.extensions/oneagent 6 6 6 6 6 beta.kubernetes.io/os=linux 18h

kubectl get deployment.extensions/dynatrace-oneagent-operator -o yaml

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

annotations:

deployment.kubernetes.io/revision: "1"

kubectl.kubernetes.io/last-applied-configuration: |

{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"dynatrace":"operator","operator":"oneagent"},"name":"dynatrace-oneagent-operator","namespace":"dynatrace"},"spec":{"replicas":1,"revisionHistoryLimit":1,"selector":{"matchLabels":{"name":"dynatrace-oneagent-operator"}},"strategy":{"type":"Recreate"},"template":{"metadata":{"labels":{"dynatrace":"operator","name":"dynatrace-oneagent-operator","operator":"oneagent"}},"spec":{"containers":[{"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}}],"image":"quay.io/dynatrace/dynatrace-oneagent-operator:v0.6.0","imagePullPolicy":"Always","name":"dynatrace-oneagent-operator","ports":[{"containerPort":60000,"name":"metrics"}],"resources":{"limits":{"cpu":"100m","memory":"256Mi"},"requests":{"cpu":"10m","memory":"64Mi"}}}],"nodeSelector":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux"},"serviceAccountName":"dynatrace-oneagent-operator"}}}}

creationTimestamp: "2020-02-20T10:00:33Z"

generation: 1

labels:

dynatrace: operator

operator: oneagent

namespace: dynatrace

resourceVersion: "15912991"

selfLink: /apis/extensions/v1beta1/namespaces/dynatrace/deployments/dynatrace-oneagent-operator

uid: d54cccb5-53c7-11ea-9bfe-4201ac100009

spec:

progressDeadlineSeconds: 600

replicas: 1

revisionHistoryLimit: 1

selector:

matchLabels:

strategy:

type: Recreate

template:

metadata:

creationTimestamp: null

labels:

dynatrace: operator

operator: oneagent

spec:

containers:

- env:

- name: WATCH_NAMESPACE

valueFrom:

fieldRef:

apiVersion: v1

fieldPath: metadata.namespace

- name: POD_NAME

valueFrom:

fieldRef:

apiVersion: v1

fieldPath: metadata.name

image: quay.io/dynatrace/dynatrace-oneagent-operator:v0.6.0

imagePullPolicy: Always

ports:

- containerPort: 60000

protocol: TCP

resources:

limits:

cpu: 100m

memory: 256Mi

requests:

cpu: 10m

memory: 64Mi

terminationMessagePath: /dev/termination-log

terminationMessagePolicy: File

dnsPolicy: ClusterFirst

nodeSelector:

beta.kubernetes.io/arch: amd64

beta.kubernetes.io/os: linux

restartPolicy: Always

schedulerName: default-scheduler

securityContext: {}

serviceAccount: dynatrace-oneagent-operator

serviceAccountName: dynatrace-oneagent-operator

terminationGracePeriodSeconds: 30

status:

availableReplicas: 1

conditions:

- lastTransitionTime: "2020-02-20T10:00:33Z"

lastUpdateTime: "2020-02-20T10:00:43Z"

message: ReplicaSet "dynatrace-oneagent-operator-87cdf56bf" has successfully progressed.

reason: NewReplicaSetAvailable

status: "True"

type: Progressing

- lastTransitionTime: "2020-02-20T10:54:48Z"

lastUpdateTime: "2020-02-20T10:54:48Z"

message: Deployment has minimum availability.

reason: MinimumReplicasAvailable

status: "True"

type: Available

observedGeneration: 1

readyReplicas: 1

replicas: 1

updatedReplicas: 1

kubectl get pod

kNAME READY STATUS RESTARTS AGE

dynatrace-oneagent-operator-87cdf56bf-894kt 1/1 Running 0 17h

oneagent-drq2r 1/1 Running 0 17h

oneagent-f76g8 1/1 Running 0 17h

oneagent-fb4ww 1/1 Running 0 17h

oneagent-fp8kp 1/1 Running 0 17h

oneagent-l2l8x 1/1 Running 0 17h

oneagent-m2v4v 1/1 Running 0 17h

kubectl logs -f dynatrace-oneagent-operator-87cdf56bf-894kt

{"level":"info","ts":"2020-02-20T10:54:48.723Z","logger":"cmd","msg":"Go Version: go1.13.6"}

{"level":"info","ts":"2020-02-20T10:54:48.723Z","logger":"cmd","msg":"Go OS/Arch: linux/amd64"}

{"level":"info","ts":"2020-02-20T10:54:48.723Z","logger":"cmd","msg":"Version of operator-sdk: v0.12.0"}

{"level":"info","ts":"2020-02-20T10:54:48.723Z","logger":"cmd","msg":"Version of dynatrace-oneagent-operator: v0.6.0"}

{"level":"info","ts":"2020-02-20T10:54:48.724Z","logger":"leader","msg":"Trying to become the leader."}

{"level":"info","ts":"2020-02-20T10:54:49.223Z","logger":"leader","msg":"Found existing lock","LockOwner":"dynatrace-oneagent-operator-87cdf56bf-m5qpc"}

{"level":"info","ts":"2020-02-20T10:54:49.235Z","logger":"leader","msg":"Not the leader. Waiting."}

{"level":"info","ts":"2020-02-20T10:54:50.364Z","logger":"leader","msg":"Not the leader. Waiting."}

{"level":"info","ts":"2020-02-20T10:54:52.747Z","logger":"leader","msg":"Not the leader. Waiting."}

{"level":"info","ts":"2020-02-20T10:54:57.287Z","logger":"leader","msg":"Became the leader."}

{"level":"info","ts":"2020-02-20T10:54:57.619Z","logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}

{"level":"info","ts":"2020-02-20T10:54:57.620Z","logger":"cmd","msg":"Registering Components."}

{"level":"info","ts":"2020-02-20T10:54:57.621Z","logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"oneagent-controller","source":"kind source: /, Kind="}

{"level":"info","ts":"2020-02-20T10:54:57.621Z","logger":"cmd","msg":"Starting the Cmd."}

{"level":"info","ts":"2020-02-20T10:54:57.621Z","logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}

{"level":"info","ts":"2020-02-20T10:54:58.022Z","logger":"controller-runtime.controller","msg":"Starting Controller","controller":"oneagent-controller"}

{"level":"info","ts":"2020-02-20T10:54:58.122Z","logger":"controller-runtime.controller","msg":"Starting workers","controller":"oneagent-controller","worker count":1}

{"level":"info","ts":"2020-02-20T10:54:58.123Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:54:58.724Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:54:58.823Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:54:58.824Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:54:58.824Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:54:58.830Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:33.777Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:33.777Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:33.790Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:36.042Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:36.043Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:36.050Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:39.261Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:39.261Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:39.270Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:39.619Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:39.620Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:39.626Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:44.282Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:44.282Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:44.296Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:50.111Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:50.112Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T10:55:50.125Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T20:54:57.723Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T20:54:58.165Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T20:54:58.170Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T20:54:58.170Z","logger":"oneagent.controller","msg":"reconciling oneagent","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T20:54:58.170Z","logger":"oneagent.controller","msg":"updating existing daemonset","namespace":"dynatrace","name":"oneagent"}

{"level":"info","ts":"2020-02-20T20:54:58.175Z","logger":"oneagent.controller","msg":"automatic oneagent update is disabled","namespace":"dynatrace","name":"oneagent"}

^C

kubectl logs -f oneagent-drq2r

10:55:00 Using volume-based storage

10:55:01 Initializing volume-based storage from within the container

10:55:01 Detected volume management CLI tool: docker

10:55:03 Started agent deployment as a container, PID 3309698.

10:55:03 Downloading agent to /tmp/Dynatrace-OneAgent-Linux.sh via https://public-ip/e/xxx-xxx-xxx-xxx/api/v1/deployment/installer/agent/unix/default/latest?Api-Token=***&arch=x86&flavor=default

10:55:03 Download was skipped as the agent version did not change

10:55:03 Validating downloaded agent installer

10:55:08 Verification successful

10:55:09 Checking root privileges...

10:55:09 OK

10:55:09 Installation started, version 1.183.163.20200203-165724, build date: 03.02.2020, PID 3311177.

10:55:09 Detected platform: LINUX arch: X86

10:55:09 Detected bitness: 64

10:55:09 Starting agents...

10:55:09 Checking root privileges...

10:55:09 OK

10:55:09 Starting Dynatrace OneAgent from within the container.

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Logger initialized, local time is: 2020-02-20 10:55

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: -----------------------------------------------------------------------------

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Dynatrace Watchdog version: 1.183.162.20200202-170257 - built 2020.02.02 17:02:57, SCM Rev. 2fc65a2385518553681f68c7b6d3924633d2a870

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Application information:

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Short name: oneagentwatchdog

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Directory: /opt/dynatrace/oneagent/agent/lib64

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Full path: /opt/dynatrace/oneagent/agent/lib64/oneagentwatchdog

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Loging information:

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Level console: 4 (info)

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Level file: 4 (info)

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Max log file age: 124 days

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Max single log file size: 64 MiB

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Filename pattern: $exename_$pid.log

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Directory: /opt/dynatrace/oneagent/log/watchdog

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Preferred log file: /opt/dynatrace/oneagent/log/watchdog/oneagentwatchdog_3309698.log

2020-02-20 10:55:09.894 UTC [89a6bd3e] info main: Log file: /opt/dynatrace/oneagent/log/watchdog/oneagentwatchdog_3309698.log

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: Invoked with command line: '/opt/dynatrace/oneagent/agent/lib64/oneagentwatchdog -vm=/opt/dynatrace/oneagent/agent/lib64/oneagentos -ini /opt/dynatrace/oneagent/agent/conf/ruxitwatchdog.ini'

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: Aging old log files

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: Memory limit calculated: 500 MiB + 5% of 7477 MiB = 873 MiB

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: Application arguments:

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: -timeout=900

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: -shutdowntimeout=60

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: -restartonfailure

2020-02-20 10:55:09.895 UTC [89a6bd3e] info main: -loglevel=4

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -native=1

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -nopassinitovm=1

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -binaries=oneagentplugin;oneagentloganalytics;oneagentnetwork

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -memorylimit.absolute=500

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -memorylimit.percentage=5

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -coredump.binary=oneagentdumpproc

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -pidfiledir=/opt/dynatrace/oneagent/agent/conf/runtime

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -log.name=$exename_$pid.log

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -logdir=/opt/dynatrace/oneagent/log/watchdog

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -portrange=50000:50100

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -basedir=.

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -vm=/opt/dynatrace/oneagent/agent/lib64/oneagentos

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: Java VM arguments:

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: -Dcom.compuware.apm.WatchDogTimeout=900

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: Java application arguments:

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: Internal settings:

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: ConfigFile=/opt/dynatrace/oneagent/agent/conf/ruxitwatchdog.ini

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: userconfigfile=/var/lib/dynatrace/oneagent/agent/config/watchdoguserconfig.conf

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: BasedirAbsolute=/opt/dynatrace/oneagent/agent/lib64

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: Unknown arguments:

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main: Watchdog initialization completed

2020-02-20 10:55:09.896 UTC [89a6bd3e] info oneagentos: Core dump generation configured to use /opt/dynatrace/oneagent/agent/lib64/oneagentdumpproc, storage path: default, timeout: 300 s

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main : Watchdog runtime configuration will be loaded from: /var/lib/dynatrace/oneagent/agent/config/watchdogruntime.conf

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main : Running in root mode

2020-02-20 10:55:09.896 UTC [89a6bd3e] info oneagentplugin: Core dump generation configured to use /opt/dynatrace/oneagent/agent/lib64/oneagentdumpproc, storage path: default, timeout: 300 s

2020-02-20 10:55:09.896 UTC [89a6bd3e] info oneagentloganalytics: Core dump generation configured to use /opt/dynatrace/oneagent/agent/lib64/oneagentdumpproc, storage path: default, timeout: 300 s

2020-02-20 10:55:09.896 UTC [89a6bd3e] info oneagentnetwork : Core dump generation configured to use /opt/dynatrace/oneagent/agent/lib64/oneagentdumpproc, storage path: default, timeout: 300 s

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main : Created pid file: /opt/dynatrace/oneagent/agent/conf/runtime/oneagentwatchdog.pid

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main : SuspensionDetector thread started

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main : Starting Watchdog, pid: 3309698

2020-02-20 10:55:09.896 UTC [89a6bd3e] info main : Signal handler configured

2020-02-20 10:55:09.897 UTC [89a6bd3e] info main : Entering main loop

2020-02-20 10:55:09.897 UTC [89a6bd3e] info main : Configuration file watchdogruntime.conf refreshed. New activation map:

2020-02-20 10:55:09.897 UTC [89a6bd3e] info main : oneagentloganalytics OFF

2020-02-20 10:55:09.897 UTC [89a6bd3e] info main : oneagentnetwork OFF

2020-02-20 10:55:09.897 UTC [89a6bd3e] info main : oneagentplugin OFF

2020-02-20 10:55:09.897 UTC [8830e87e] info oneagentos : Start main thread

2020-02-20 10:55:09.897 UTC [87b0f87e] info oneagentos : Trying to open port 50000

2020-02-20 10:55:09.897 UTC [87b0f87e] info oneagentos : Watchdog successfully opened listening port 50000

2020-02-20 10:55:09.897 UTC [87b0f87e] info oneagentos : BinaryWatcher thread started successfully, listening on socket 50000

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : BinaryWatcher initialized

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Entering application restart loop

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : -----------------------------------------------------------------------------

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Starting process

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : -----------------------------------------------------------------------------

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Launcher configuration summary for oneagentos

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Binary: /opt/dynatrace/oneagent/agent/lib64/oneagentos

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Working directory: /opt/dynatrace/oneagent/agent/lib64

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Native watchdog enabled, port: 50000, timeout: 900 s (timeout for graceful shutdown: 300 s)

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Memory usage limit: 873 MiB

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Command line: /opt/dynatrace/oneagent/agent/lib64/oneagentos -Dcom.compuware.apm.WatchDogPort=50000 -Dcom.compuware.apm.WatchDogTimeout=900

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Custom environment variables:

2020-02-20 10:55:09.918 UTC [8830e87e] info oneagentos : Process created successfully, pid: 3311904

2020-02-20 10:55:09.919 UTC [8830e87e] info oneagentos : BinaryWatcher activated

2020-02-20 10:55:09.919 UTC [8830e87e] info oneagentos : Starting to wait (blocking) for waitpid return, pid number = 3311904

2020-02-20 10:55:15.899 UTC [87b0f87e] info oneagentos : New connection on port 50000 received.

2020-02-20 10:55:15.899 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 1

2020-02-20 11:05:16.336 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 61

2020-02-20 11:15:17.704 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 121

2020-02-20 11:25:19.074 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 181

2020-02-20 11:35:20.412 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 241

2020-02-20 11:45:21.877 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 301

2020-02-20 11:55:23.255 UTC [87b0f87e] info oneagentos : Ping received: [HEALTHY:3311904] observedPid: 3311904, total ping count: 361

skrystosik · ‎21 Feb 2020

You see hosts in dynatrace? Or you don't see hosts / transaction & services?

Sebastian

Regards, Sebastian

salih_pala · ‎21 Feb 2020

Hello Sebastian,

No, ı'm not see hosts / transaction & services for my nodes.

skrystosik · ‎21 Feb 2020

And you are sure that your Kubernetes nodes have open network traffic to ActiveGate? (It's different communication than between ActiveGate and Kubernetes cluster for metrics collection).

Sebastian

Regards, Sebastian

salih_pala · ‎21 Feb 2020

It did not have access to ports 8443 and 9999. I'll try again by defining access permissions. Could that be the cause of the problem ?

But if there was an access problem, the pods would log the connection-related error.

I did not understand.

skrystosik · ‎21 Feb 2020

For ActiveGate should be 9999 (OneAgent => AG) from AG to DT Server 443/8443

Regards, Sebastian