AWS fargate

[Cath]

Hi,

We have some spring boot application, running in a docker container with AWS fargate.

We used the runtime injection described here https://docs.dynatrace.com/docs/ingest-from/amazon-web-services/integrate-into-aws/aws-fargate

Everything was working fine. 

That is, until security showed up, saying we shouldn't use "root" to start the container. 

So add added the user in the dockerfile, made the proper chown / chmod to /opt/dynatrace. The first part goes well, downloading the agent, unzipping... Then, when the app starts, one agent isn't started. No initialization logs, no error logs. Just nothing. Note that DT_LOGLEVELCON is set to FINEST

Removing USER my_user from the docker file fix it, but don't fix the security issue.

Yet no matter what I do, no monitoring unless I start as root.

At this point I'm thinking the only way out is to switch to build-time injection. 

Any clues, what could a possible be missing ?

[Julius_Loman]

You have either permission issues with loading oneagent (but you write you did a chown/chmod) or your preloading applies to root user - which is most likely. Try to add ENV LD_PRELOAD command from the docs for the user - meaning after the USER statement in the Dockerfile.

[Cath]

Tanks. 

Did try that. But no chance. 

Though about some missing capacities, but my tries there where unsuccessful as well.  

I went into the container to manually call
dynatrace-agant.sh java my_app.jar

as root, you see the one agant logs. As my custom user, you see using DT_HOME ... then the process seams to hang for a little while, and the app starts. 

[Julius_Loman]

Can you share how does your Dockerfile look like? And can you show the runtime configuration of the container? Env variables / command?

Looking for the method how OneAgent gets loaded in your case.

[Cath]

RUN mkdir -p /opt/dynatrace/oneagent && chown -R myuser:myuser/opt/dynatrace && chmod -R 777 /opt/dynatrace

USER myuser
VOLUME ["/opt/dynatrace/oneagent"]

ENV DT_DEBUG="true"
ENV DT_LOGLEVELCON="DEBUG"
ENV LD_PRELOAD="/opt/dynatrace/oneagent/agent/lib64/liboneagentloader.so"

ENTRYPOINT ["/bin/sh", "./docker_args.sh"]

where docker_args.sh do java -jar myapp.jar

Originally it was also in the task definition with the same values. 

[Julius_Loman]

@Cath are you using the runtime injection? https://docs.dynatrace.com/docs/shortlink/aws-fargate#runtime 

Based on your Dockerfile I belive you don't have correct permissions on the OneAgent files in the volume you are mounting.  Can you check that if you run a shell in your container? 

If you can run a shell for myuser user, just run:

LD_PRELOAD="/opt/dynatrace/oneagent/agent/lib64/liboneagentloader.so" java -jar myapp.jar

 It should give an stderr error if library cannot be preloaded.

[Cath]

Ok. your remark got me to try some stuff with export LD_PRELOAD= ..

and it seam in some situation, LD_PRELOAD is just... ignored.

https://unix.stackexchange.com/questions/757484/ld-preload-does-not-work-and-ld-debug-shows-nothing

I did add capabilities. 

Don't fix the trouble yet, but at least it a clue.

[Cath]

Soo. 

I replaced the 777 by +x, removed capacities on the java process. 

Our app don't start, can't bind port 80. But, dynatrace agent starts. All along we where looking for not enough permission. It was a problem of too many permissions triggering linux security. 

[Julius_Loman]

@Cath yes of course it did not start as it was already started in the container. So your permissions in the volume with OneAgent data must be fixed.