Dynatrace set up on AWS

[agrawal_shashan]

Hello, I am trying to set up Dynatrace to monitor resources in AWS. We use Dynatrace Managed which is hosted on our on-prem infrastructure so following this documentation, we have set up an Environment Active gate on EC2 instance.

I have completed all the steps listed, created an IAM role and attached to the EC2 instance where my Environment Active gate is deployed. Now while doing the last step which is listed here when I make the connection to AWS from Dynatrace UI, I am getting an error message which says "Active gate unavailable" (screenshot attached). I am not able to understand why? I've checked and my env active gate is up and running.

Also how does the flow work? Dynatrace AWS pushes the metrics to Dynatrace or is it Dynatrace which pulls the metrics from AWS?

Any help on this is really appreciated.

Best Regards,
Shashank

[victor_balbuena]

Hey @agrawal_shashan ,

Is it possible that the ActiveGate you're set up is not correctly linked to your tenant, as in, there is no communication to it somehow? Does it appear if you search for it under Deployment Status -> ActiveGates? And does it have the AWS module enabled?

For your second question, it is the ActiveGate itself that connects to your AWS account, polls the metrics from AWS Cloudwatch and then sends them to the Dynatrace cluster - everything happens in the ActiveGate.

[agrawal_shashan]

Hi @victor_balbuena Thanks for the response. So right now I have an EC2 instance in a AWS account (XYZ) where I have also deployed Dynatrace Active gate. This EC2 instance has connectivity open to our Dynatrace Managed Cluster.

And in Dynatrace UI also I am just trying to connect to this same AWS account (XYZ) for now but it gives me that error which I pasted. Just trying to understand when I click on connect, what happens? Does Dynatrace managed cluster tries to connect to AWS or is it Env Active gate on AWS tries to pull the metrics from the same account?

FYI.. AWS module is enabled on the Env Active gate.

[agrawal_shashan]

Hey @victor_balbuena I was actually connecting from wrong Dynatrace Env but I rectified it and now trying from the correct tenant/env. But now I am getting a different error which says "Invalid Credentials".

Also below are the logs from Env Active gate -

2023-10-02 09:09:34 UTC INFO    [<XXXXXXX-XXXXXXXX-XXXXXX>] [<vtopology.provider>, RoleCredentialsProvider] Cannot obtain CLIENT short term credentials for arniam::XXXXXXXXXXXX:role/Dynatrace_ActiveGate_role ; AWSCredentialsImpl {identifier: XXXXXXXX, accessKey: null, secretKey: null, tenantUUID: XXXXXXX-XXXXXXXX-XXXXXX, iamRole: Dynatrace_ActiveGate_role, accountId: XXXXXXXXX, externalId: *****, label: Dynatrace Integration, partition: aws, detectedPartition: aws, monitorOnlyTaggedEntities: false, includeTags: [], excludeTags: [], excludedRegions: [], logConfigSQSesEnabled: false, logConfigSQSes: [], version: 2.0, legacyServices: [ebs_builtin, lambda_builtin, ELB_builtin, loadbalancer_builtin, s3_builtin, dynamodb_builtin, ec2_builtin, asg_builtin, rds_builtin], services: []} [Suppressing further identical messages for 10 minutes]
com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to sts.amazonaws.com:443 [sts.amazonaws.com/209.54.180.124] failed: connect timed out
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1219)

2023-10-02 09:09:34 UTC WARNING [<XXXXXXX-XXXXXXXX-XXXXXX>] [<vtopology.provider>, AWSFastCheckCallable] Credentials refresh failed: {status: ERROR_BAD_CREDENTIALS, statusInfo: Service failed to assume role provided in credentials, credentials: AWSCredentialsImpl {identifier: XXXXXXXXX, accessKey: null, tenantUUID: XXXXXXX-XXXXXXXX-XXXXXX, iamRole: Dynatrace_ActiveGate_role, accountId: XXXXXXXX, externalId: *****, label: Dynatrace Integration, version: 2.0}, exception: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to sts.amazonaws.com:443 [sts.amazonaws.com/209.54.180.124] failed: connect timed out}

[victor_balbuena · Accepted solution]

When you click on connect, it's the ActiveGate reaching out to test the connection to AWS, so it acknowledges the connection works before it's set up. Dynatrace Managed is not involved in this step. Once it is set up, the ActiveGate will try to send the data to Dynatrace Managed, but Dynatrace Managed does not reach out to any resource ever.

As per the issue, we are falling into AWS teritory now, so it might make more sense if some expert from AWS takes a look or you talk to Dynatrace support directly. Having said that, something you can look into is the outbound security rules of your EC2 instance (where the ActiveGate is running), to allow for requests and data to leave the ActiveGate.

[agrawal_shashan]

Hi @victor_balbuena Your information has been immensly helpful. Thank you very much.

Again looking at this documentation it says "Make sure that your Environment ActiveGate or Managed Cluster has a working connection to AWS. Configure your proxy for Managed or ActiveGate, or allow access to *.amazonaws.com in your firewall settings." 

And in the logs I can see its trying to make a connection to 

sts.amazonaws.com:443

but failing. Trying to understand if it is the Active gate which tries to make this connection?

Best Regards,
Shashank

[victor_balbuena]

Yes, it is the ActiveGate in this case 😊

[Pawel_Zalewski]

Hi Agrawal,

Did you change MonitoringRoleName after upload YAML file from github role_based_access_monitored_account_template.yml in Stack Details?

In your screenshot I see in field "IAM role that Dynatrace should use to get monitoring data":

Dynatrace_ActiveGate_role

but in default is:

Dynatrace_monitoring_role

Best Regards

Paweł