Kubernetes OneAgent-Deploy Init Container Failure

Platform Container platforms
, ,
1

We’re using cloud native full stack on our linux nodes and pods. 
We use the OneAgent-Deploy init container to instrument our node application. 
Suddenly our newly created nodes are failing to instrument the applications in their pods. 
There is an error in the init container logs saying.

“level”:“info”,“ts”:“2024-02-14T05:07:37.135Z”,“logger”:“standalone-init”,“msg”:“standalone agent init started”}

{“level”:“info”,“ts”:“2024-02-14T05:07:37.135Z”,“logger”:“standalone-init”,“msg”:“setting host tenant”}

{“error”:“host tenant info is missing for aks-userreact-80605415-vmss0000x0”,“level”:“error”,“logger”:“standalone-init”,“msg”:“This error has been masked to not fail the container.”,“stacktrace”:"host tenant info is missing for aks-userreact-80605415-vmss0000x0

It seems the secret config value that is printed in the log contains a key value hash of nodes to dynatrace tenant. 
This secret doesn’t contain the newly created node name in it. 
eg.

{“level”:“info”,“ts”:“2024-02-14T05:07:37.135Z”,“logger”:“standalone-init”,“msg”:“contents of secret config”,“content”:{…

“aks-user-42685077-vmss00004b”:“twh49101”…

What is it that populates this key value pair in the config secret? 
What could cause this issue?

2

We found the root cause and solved the issue.
The issue was that our azure app registration secret that was being used for the dynakube-pull-secret secret in the dynatrace kubernetes namespace had expired.
This was causing the dynatrace operator to fail pulling from our ACR and for reasons stopped it from updating the dynatrace-dynakube-config secret in our instrumented namespaces from being updated. 
Once we updated secret, injected it into the dynatrace namespace and restarted the operator pod the secret in our instrumented namespaces got updated with new node info.
Restarting our workloads then resulted in our application being instrumented.