topic Re: Log Monitoring - DQL in DQL

Log Monitoring - DQL

munawarhossain1 — Tue, 06 Feb 2024 13:46:51 GMT

Hello,

I'm using this query and I want to exclude a particular text from logs -

"fetch logs //, scanLimitGBytes: 500, samplingRatio: 1000
| filter matchesValue(dt.entity.process_group, "abcd") and matchesValue(status, "ERROR")"

Please let me know, how?

Thanks in advance.

Re: Log Monitoring - DQL

gbaudart — Tue, 06 Feb 2024 18:59:01 GMT

HI @munawarhossain1,
What text would you like to remove?
To make things clearer, can you show us the results of your query pls?

Re: Log Monitoring - DQL

munawarhossain1 — Wed, 07 Feb 2024 05:20:48 GMT

Sure @gbaudart

"Content

Feb 7 05:13:42 abcd python: 2024/02/07 05:13:42.738654 INFO ExtHandler cached endpoint not found"

I want to remove "ExtHandler cached endpoint not found" from the all the logs available for a particular host/HG".

I know that's doable via ingestion rule, but I'm focused to see if this can be done via my above query.

Thanks

Re: Log Monitoring - DQL

bsnurka — Wed, 07 Feb 2024 18:07:50 GMT

Hey @munawarhossain1 , would using the filterOut cmd be viable? I am assuming you want the DQL query to return all logs that matches the first two conditions combined via AND, but specifically exclude all instances of "ExtHandler cached endpoint not found"

fetch logs | filter matchesValue(dt.entity.process_group, "abcd") and matchesValue(status, "ERROR") | filterOut contains(content, "ExtHandler cached endpoint not found")

Re: Log Monitoring - DQL

munawarhossain1 — Thu, 08 Feb 2024 08:23:14 GMT

That's work mate, thanks!

But as ingesting rule is defined to not capture content "bulk" but it is popping now it seems into logs

Could you please help me to rectify the condition?

Thanks

Re: Log Monitoring - DQL

bsnurka — Thu, 08 Feb 2024 16:07:54 GMT

Is it as simple as the rule being defined as "Bulk" in the second Log content parameter but our actual log content is "bulk"?

Re: Log Monitoring - DQL

munawarhossain1 — Thu, 08 Feb 2024 16:14:00 GMT

I've tweaked that now -

But still appearing -

Re: Log Monitoring - DQL

bsnurka — Thu, 08 Feb 2024 16:21:53 GMT

I am under the assumption your goal is to exclude the ingestation of any logs that have "This bulk request has failures failure in bulk execution:" in its log content. Assuming so, you will need to look into the Log Ingest Rules. There are some specifics around the wildcards accepted by Dynatrace, which I think the example from the documentation illustrates well:

If no wildcard is used in the value, then the matcher looks for an exact fit to the value. If a wildcard is used, the matcher looks for the exact match. For example, the value INFO results in sending only the log data having the exact INFO string, but the value *INFO* (using the wildcards) matches log data that contain the INFO string in its content.

Right now based off the screenshots, I suspect your log ingest rule is looking for log content that simply says "bulk" or "Bulk" as we aren't configured with a wildcard.

Re: Log Monitoring - DQL

munawarhossain1 — Thu, 08 Feb 2024 16:27:22 GMT

Alright thanks, changes are made now -

I'll wait for a result now