Ideal or recommended way of monitoring processes in container

[rrcamara]

We are using Dynatrace Oneagent agent in Amazon Linux 2 EC2 instance. It is installed as a standard package. Our developers have started migrating their applications to docker containers. Some of their applications are now running inside a docker container. I noticed lately that our developers have asked us to restart their containers. It's because they saw something in the Dynatrace console saying "Restart the container". Here is the exact message that they sent to me

Why is this happening? I'm wondering why this message happened even if their docker container started after oneagent started during a fresh boot-up.

[PierreGutierrez]

Hi @rrcamara 

The message you show us indicates that the process instance must be restarted, so that it can be well monitored at the Full-Stack level, and you can obtain all the information that this modality provides you. 

It's true that this message appears occasionally when the instance is generated before the OneAgent process, however, in my experience, there are other possible reasons why it also shows the message:

- The instance had problems getting up.
- The instance node could be overloaded with resources.

In conclusion, if the agent fails to inject the code correctly for deep-monitoring, a message may be displayed indicating the restart of the instance.

I hope it's helpful 💪

[rrcamara]

However, when I execute "systemd-analyze critical-chain", it shows that the docker container process started after oneagent has started.

[PierreGutierrez]

Hi @rrcamara 
Maybe you can share more information with us, a complete image of what processes, technology, resources in node/pod.

Maybe it would be useful to understand it better.

[rrcamara]

We are running Docker daemon in EC2 instance. It's a java application running inside a docker container. The container is created using docker-compose command. The host machine is Amazon Linux 2. Please let me know if I missed answering anything from your question.

[Esam_Eid]

I have seen this message even after restarting the pods/process because the technology is not supported 

[rrcamara]

Do you mean, Dynatrace OneAgent doesn't support monitoring of processes that are inside the container?

[Esam_Eid]

could be the reason, what is the process technology ?

[rrcamara]

java application

[PacoPorro]

Are you using https://docs.dynatrace.com/docs/shortlink/aws-distributor to deploy the oneagent?

[rrcamara]

No. I built my own unix shell script to install Oneagent. It is working well until some of our apps got migrated to docker containers.

[PacoPorro]

You face a "race condition" and your application starts before oneagent.
Would it be possible to use the aws-distributor at least for a test?