How or when can I associate Kuberenetes, PG, or AWS data to a log record in openpipeline?

[ronmacdonald]

There is meta-data on those entities that would be very useful to be incorporated into a log record.  I understand DQL allows for joins and similar functions post processing, but this increases the complexity of DQL, inhibits adoption, and increases query expenses.

[jeffscelza76]

I was informed that Native K8s starts rolling out today with Agent 1.305. This should retain a lot of metadata that Fluent-bit tends to strip out and provide better integration with other components in Dynatrace.   Our SE's did a test within their environment and their finding was that it did in fact provide that metadata (i.e. process.technology, pod info, etc) that our pipeline are set up to use. Once out, we plan on migrate within a few days on our lower environments. I can report back or you can reach out if you want to know what the results are.

https://docs.dynatrace.com/docs/whats-new/release-notes/oneagent/sprint-305#new-features-and-enhance... 

Also, in 1.305 SaaS release notes there was some breaking changes regarding how logs are handled in AWS.  We haven't started migrating our AWS logs from our current solution so I can't not speak to that personally.  

https://docs.dynatrace.com/docs/whats-new/release-notes/saas/sprint-305 

Hopefully, this helps. 

[StefanPostma]

Within Kubernetes the Dynatrace operator includes a feature called meta data enrichment. 
https://docs.dynatrace.com/docs/shortlink/k8s-metadata-telemetry-enrichment
This includes adding meta data from the source of the logs file. 

Another feature you might want to look into is log enrichment feature from the oneAgent. This allows the oneAgent to add dt_trace_id to be added to the logline while the log is written. After that is does not matter how the log is shipped to Dynatrace as the logline containts the trace ID. : https://docs.dynatrace.com/docs/shortlink/lma-log-enrichment#automatic