Log Monitoring - DQL

[munawarhossain1]

Hello,

I'm using this query and I want to exclude a particular text from logs -

"fetch logs //, scanLimitGBytes: 500, samplingRatio: 1000
| filter matchesValue(dt.entity.process_group, "abcd") and matchesValue(status, "ERROR")"

Please let me know, how?

Thanks in advance.

[gbaudart]

HI @munawarhossain1,
What text would you like to remove?
To make things clearer, can you show us the results of your query pls?

[munawarhossain1]

Sure @gbaudart 

"Content

Feb 7 05:13:42 abcd python: 2024/02/07 05:13:42.738654 INFO ExtHandler cached endpoint not found"

I want to remove "ExtHandler cached endpoint not found" from the all the logs available for a particular host/HG".

I know that's doable via ingestion rule, but I'm focused to see if this can be done via my above query.

Thanks

[bsnurka · Accepted solution]

Hey @munawarhossain1 , would using the filterOut cmd be viable? I am assuming you want the DQL query to return all logs that matches the first two conditions combined via AND, but specifically exclude all instances of "ExtHandler cached endpoint not found"

fetch logs
| filter matchesValue(dt.entity.process_group, "abcd") and matchesValue(status, "ERROR")
| filterOut contains(content, "ExtHandler cached endpoint not found")

[munawarhossain1]

That's work mate, thanks!

But as ingesting rule is defined to not capture content "bulk" but it is popping now it seems into logs

Could you please help me to rectify the condition?

Thanks

[bsnurka]

Is it as simple as the rule being defined as "Bulk" in the second Log content parameter but our actual log content is "bulk"?

[munawarhossain1]

I've tweaked that now - 

But still appearing -

[bsnurka]

I am under the assumption your goal is to exclude the ingestation of any logs that have "This bulk request has failures failure in bulk execution:" in its log content. Assuming so, you will need to look into the Log Ingest Rules. There are some specifics around the wildcards accepted by Dynatrace, which I think the example from the documentation illustrates well:

---

If no wildcard is used in the value, then the matcher looks for an exact fit to the value. If a wildcard is used, the matcher looks for the exact match. For example, the value INFO results in sending only the log data having the exact INFO string, but the value *INFO* (using the wildcards) matches log data that contain the INFO string in its content.

---

Right now based off the screenshots, I suspect your log ingest rule is looking for log content that simply says "bulk" or "Bulk" as we aren't configured with a wildcard.

[munawarhossain1 · Accepted solution]

Alright thanks, changes are made now -

I'll wait for a result now