Showing results for 
Show  only  | Search instead for 
Did you mean: 

Log source value in Dynatrace when using Wildcards


Fairly new to log monitoring, we are doing a POC with it. Anyone find it strange you cannot see the EXACT filename unless you actually setup the custom log source configuration rule to have an exact path (aka not using wildcards).

Let's say I have any number of directories (the list is never static) within the folder structure below. Within those directories I have a file by the name of <some_name.log> I could do something like /logs/*/*/*.log to capture all of these files but then when I query within logs and events in dynatrace, I see the log.source value as the custom log source rule matcher I setup, in this example '/logs/*/*/*.log'.

I thought the whole purpose of a log monitoring tool was to be able see the exact file the issue occurred in but instead with dynatrace I would have to still logon to the host and do my own greps to find the file where the message was logged from. I'll also add that yes you can automate the log source rules via api calls (to create the exact matcher) but are we really expected to have potentially hundreds of matchers and then also have to worry about when a new directory is spun up, adding that new matcher?

I could very well be missing something here, looking for feedback here.

Example folder structure below. Note, within each directory can be any number of directories which within them, could have any number of log files.



Dynatrace Champion
Dynatrace Champion

At the moment, this is "by design" apparently. I also have a customer struggling with this (they're just using the '#' wildcard in their filenames but also would like to know the concrete filename the entry was pulled from).
I'd be curious whether anyone else has an idea.

We have encountered similar issues and disappointment with this.  What completely stumps me though is why the documentation for defining custom log sources under Log File Matching states "Custom log sources can contain wildcards..." when using them may very well break your ability to search the data you just defined for ingestion.  If you use a * anywhere in the middle of the path to the log file the UI will provide a link to your newly defined file as expected.  But here's the challenge, you can't search this new log source.  When you try the UI presents the following message: "Unexpected error: Invalid DQL query: [2,10] MATCHES_VALUE: Wildcard '*" in the middle is not supported".  This can be worked around by playing with the advanced search, but I can't imagine explaining this to everyone at the company that wants to search their data.  I suppose another option/workaround might be fully qualifying the path and filename (removing the wildcard altogether) for every log source that's not automatically discovered but yikes!  

I'm going to assume this is all just something I've not given enough thought to yet because I know Dynatrace is better than this.  🙂

Hi Alvin,
there are two issues present here:
1. If you have custom log sources with wildcards, these wildcards are used to match files, but they are not resolved on UI, i.e. log source name still has wildcards. Shortly speaking, this is good for some use cases, and bad for other ones. Having acknowledged these shortcomings, we came up with an idea to allow for decorating log records with an additional attribute carrying the origin file name. This is planned for CQ4'23. The solution is decided so there is a high chance to deliver it as planned.

2. You cannot use a '*' character in the middle of a log query. This character is interpreted always as a wildcard (which is not the intention here but would work in most cases) and the wildcard is allowed only at the beginning and at the end of matched value. We have a story to provide a possibility to mask the wildcard character to match it literally which would cover your use case, but unfortunately we do not have capacity to implement it in CQ4'23, at least for now. A workaround is possible in most cases. You can use "'prefix*' and '*suffix'" instead of "`prefix*suffix'" condition. I am aware that is not very elegant.

Hi @Joachim_Erdei 
I am very interested in this additional attribute with the original file name.
Will it be delivered as planned? More precise information on the delivery date ?

Dynatrace Contributor
Dynatrace Contributor

Hi Gerard, this is already in progress. I estimate that it should be ready in OneAgent 1.285, Cluster 1.286. Even if I misjudge the effort here, a slight move in any direction will still land in CQ1'24 - no promise here, but I am quite certain about the date personally.

Is this expected to be released soon? We are on cluster version 1.288, hoping to see this improvement soon.

Dynatrace Contributor
Dynatrace Contributor

Hi, unfortunately it slipped a little, but it is going to be available in Cluster 1.291

No worries, thank you for the update.

@Joachim_Erdei did this get pushed again? We are on cluster 1.291 and I've updated to the latest agent version. I don't see any change here and didn't see anything noted in the 1.291 release notes.

Dynatrace Contributor
Dynatrace Contributor

It is already done but it will be released in Cluster 1.292 (Agent 1.285 required). Sorry for one more delay, but it was the last one - the functionality is finished and tested and just waiting to be released.

No problem, looking forward to this one. Thank you.

Featured Posts