topic Re: Enriching logs with AWS entity information in Log Analytics

Enriching logs with AWS entity information

manisundaram — Thu, 19 Dec 2024 07:39:21 GMT

we have 2 integrations in place

1. AWS Cloudwatch integration, where we are pulling in cloudwatch metrics for various AWS Services

2. Log integration. We are pulling in AWS Database Migration service logs from cloudwatch into Dynatrace

When logs are coming in, they are not getting mapped to entities (AWS DMS Entities)

When looking through documentation for Dynatrace, entity linking is not available for DMS Services - https://docs.dynatrace.com/docs/shortlink/lma-stream-logs-with-firehose#view-aws-logs

We are thinking if we can use OpenPipeline to achieve this ? Is it possible - any thoughts ?

Re: Enriching logs with AWS entity information

RohitBisht — Thu, 19 Dec 2024 02:58:30 GMT

Hi Mani,

It is possible you can get the "entity id" from the AWS service and manually tag the logs using classic log processing rules or openpipeline.
I have done something similar in the past for cloudwatch logs and it works well.
Best is to get details about the AWS service entity from the API and then reuse it to map the topology accordingly.
https://docs.dynatrace.com/docs/shortlink/api-entities-v2-get-entity

Re: Enriching logs with AWS entity information

Peter_Youssef — Thu, 19 Dec 2024 06:27:06 GMT

Hi @manisundaram

In addition to valuable inputs by @RohitBisht

Regarding the raised concern PFA resources for actual execution as per monitoring requirements:

AWS:

Entities:

https://docs.dynatrace.com/docs/discover-dynatrace/references/dynatrace-api/environment-api/entity-v2

Logs:

Log processing with Open Pipeline:

https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-openpipeline

Regarding the accurate mapping:

You can refer to the available topology relationships upon the successful integration.
https://docs.dynatrace.com/docs/discover-dynatrace/references/semantic-dictionary/model/dt-entities

Hoping it helps.

BR,

Peter