cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

s3 log forwarding

rgarzon1
Mentor

I am trying to use this for ingests logs 

 github.com/dynatrace-oss/dynatrace-aws-s3-log-forwarder

resume

export STACK_NAME=dynatrace-aws-s3-log-forwarder
export DYNATRACE_TENANT_UUID=*************
export PARAMETER_NAME="/dynatrace/s3-log-forwarder/$STACK_NAME/$DYNATRACE_TENANT_UUID/api-key"
export HISTCONTROL=ignorespace
export PARAMETER_VALUE=********************
aws ssm put-parameter --name $PARAMETER_NAME --type SecureString --value $PARAMETER_VALUE
aws ecr create-repository --repository-name dynatrace-aws-s3-log-forwarder
export VERSION_TAG=$(curl -s https://{environmentid}.live.dynatrace.com" \
            DynatraceEnvironment1ApiKeyParameter=$PARAMETER_NAME \
            ContainerImageUri=${REPOSITORY_URI}:${VERSION_TAG}-x86_64 \
            --template-file template.yaml --capabilities CAPABILITY_IAM 
            
// until here, everything works fine ( except for some errors that appear duplicate function, low concurrency that were resolved) 

//optional
aws cloudformation deploy --stack-name $STACK_NAME --parameter-overrides \
    EnableCrossRegionCrossAccountForwarding=true \
    --template-file template.yaml --capabilities CAPABILITY_IAM             
            

//validating
echo $STACK_NAME
echo $PARAMETER_NAME
echo $PARAMETER_VALUE
echo $VERSION_TAG
echo $REPOSITORY_URI

//configuration of forwarder configuration
bucket_name: logsdynatrae
log_forwarding_rules:
  - name: log_generic         
    prefix: ".*"      
    source: generic
    annotations:
      environment: dev
      
aws cloudformation deploy \
    --template-file dynatrace-aws-s3-log-forwarder-configuration.yaml \
    --stack-name dynatrace-aws-s3-log-forwarder-configuration-$STACK_NAME \
    --parameter-overrides DynatraceAwsS3LogForwarderStackName=$STACK_NAME
            
//eventbridge configuration
aws s3api put-bucket-notification-configuration --bucket logsdynatrae --notification-configuration='{ "EventBridgeConfiguration": {} }'


export BUCKET_NAME=logsdynatrae

aws cloudformation deploy \
    --template-file dynatrace-aws-s3-log-forwarder-s3-bucket-configuration.yaml \
    --stack-name dynatrace-aws-s3-log-forwarder-s3-bucket-configuration-$BUCKET_NAME \
    --parameter-overrides DynatraceAwsS3LogForwarderStackName=$STACK_NAME \
                          LogsBucketName=$BUCKET_NAME \
    --capabilities CAPABILITY_IAM

 

and next loading to the s3 a log for testing purposes with the next information

logs3testing_01.txt

[2024-04-11 20:39:43.752 UTC] [aws] [debug] For Logs in s3

 

this can be done in 15 minutes, but after 6 hours testing deploying, understanding, deploying, updating. understanding, validating,  get nothing

i am doing something wrong ?

 

fuelled by coffee and curiosity.
1 REPLY 1

rgarzon1
Mentor

I found my error, the bucket was in another region. I want to cry haha 😭

fuelled by coffee and curiosity.

Featured Posts