topic Re: Limit Resource for OneAgent DeamonSet in Container platforms

Limit Resource for OneAgent DeamonSet

xuyenpn — Wed, 15 Nov 2023 13:45:42 GMT

Hi all,
We are integrating dynatrace with k8s version 1.20 using DeamonSet.

I see in the yaml file there is a resource limitation for Operator. What about DeamonSet?

Re: Limit Resource for OneAgent DeamonSet

eduard_van_der1 — Wed, 15 Nov 2023 08:25:20 GMT

Hi,

You should be able to define the resource limits as we are deploying a DaemonSet (Not a custom resource like the operator).

An example of limiting resources can be found here in the Kubernetes documentation: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/

200Mi is probably not enough for the OneAgent - I'd go with similar limits that are recommended for the Operator.

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Wed, 15 Nov 2023 08:55:38 GMT

I try to it. But All Oneagent Pods don't working.

OneAgent Pods status is: CrashLoopBackOff

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Thu, 16 Nov 2023 07:25:33 GMT

My dynatrace-oneagent.yaml:

apiVersion: apps/v1
kind: DaemonSet
metadata:
name: dynatrace-oneagent
spec:
selector:
matchLabels:
name: dynatrace-oneagent
template:
metadata:
labels:
name: dynatrace-oneagent
spec:
hostPID: true
hostIPC: true
hostNetwork: true
nodeSelector:
beta.kubernetes.io/os: linux
volumes:
- name: host-root
hostPath:
path: /
containers:
- name: dynatrace-oneagent
image: dynatrace/oneagent
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
env:
- name: ONEAGENT_INSTALLER_SCRIPT_URL
value: https://env/e/700/api/v1/deployment/installer/agent/unix/default/latest?arch=<arch>
- name: ONEAGENT_INSTALLER_SKIP_CERT_CHECK
value: 'false'
- name: ONEAGENT_INSTALLER_DOWNLOAD_TOKEN
value: ''
args:
- '--set-network-zone=<your.network.zone>'
volumeMounts:
- name: host-root
mountPath: /mnt/root
securityContext:
privileged: true

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Thu, 16 Nov 2023 07:25:08 GMT

My dynatrace-oneagent.yaml:

Re: Limit Resource for OneAgent DeamonSet

eduard_van_der1 — Wed, 15 Nov 2023 08:57:48 GMT

Hi, What is the crash reason? Very likely it runs out of memory at 200Mi - try to increase it to 500Mi or 1G

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Wed, 15 Nov 2023 09:10:05 GMT

Oh, I try to increase it to 500M but not running. I will try to increase to more memory. Lol

Re: Limit Resource for OneAgent DeamonSet

eduard_van_der1 — Wed, 15 Nov 2023 09:11:28 GMT

You can describe the Pod to see what caused the failure.

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Wed, 15 Nov 2023 09:21:25 GMT

I see OneAgent Pods status is: CrashLoopBackOff

I try to get event and result as picture:

Re: Limit Resource for OneAgent DeamonSet

eduard_van_der1 — Wed, 15 Nov 2023 09:29:00 GMT

Thanks. This indicates that either the container host operating system is running out of memory.

The Linux OOMKiller will designate a victim (in this case one of the processes from the OneAgent pods), and that causes the CrashLoopBackoff. So you have to check that your host has sufficient memory available - and then make sure that the limit for the OneAgent is high enough.

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Wed, 15 Nov 2023 09:48:10 GMT

The K8s cluster have 3 master node & 3 worker node corresponding to 3 oneagent daemon. I see that 1 oneagent using a small amount of memory as picture:

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Thu, 16 Nov 2023 01:42:04 GMT

Sorry. Noted that: I use deamonset for deployment oneagent

Re: Limit Resource for OneAgent DeamonSet

Romanenkov_Al3x — Thu, 16 Nov 2023 06:34:41 GMT

Hello @xuyenpn , removed full token from your reply.

The limits are best set as defined by the Dynatrace operator's developers.

On the github, you can select the desired version and see the default limits in the example file dynakube

oneAgentResources: requests: cpu: 100m memory: 512Mi limits: cpu: 300m memory: 1.5Gi

You can easly check it for different versions in samples of dynakube.

You can find recommended operator version from documentation page.

Regards,

Alex Romanenkov

Re: Limit Resource for OneAgent DeamonSet

xuyenpn — Mon, 20 Nov 2023 02:10:31 GMT

Thanks for your respond.
I removed full token from my reply.

Noted. I try to using DaemonSet for K8s version 1.16. Don't using Operator