topic Re: "My Event" Problem Alerts in Alerting

"My Event" Problem Alerts

ASE — Thu, 10 Oct 2024 18:25:58 GMT

We recently started seeing Problem Alerts in our Problems app titled "My Event".

I'm having trouble determining where they are coming from, as they do not provide much information.

Any ideas on what could be generating these alerts?

Thank you.

Re: "My Event" Problem Alerts

p_devulapalli — Fri, 11 Oct 2024 02:04:16 GMT

Hi @ASE , did you try to check if you have anything configured under Anomaly detection-->Metric events that's triggering this alert ?

Re: "My Event" Problem Alerts

ASE — Fri, 11 Oct 2024 14:36:14 GMT

Yes, @p_devulapalli.

We have 250+ Metric Events configured.

I don't see any custom Metric Event that I'm not expecting to see.

Don't you need an actual metric in order to create a Metric Event?

When you look at the Problem Alert I shared, there is no sign of a metric.

This comes across as something built-in (out of the box), but I can't seem to find what that could be.

I know we can create custom Log Events and custom Metric Events. Other than these 2 places, I don't know where else it could be coming from.

Re: "My Event" Problem Alerts

Peter_Youssef — Fri, 11 Oct 2024 15:31:22 GMT

Hi @ASE

Kindly Check:

If there are integrations with external entities where is no way to install OA, so it might be ingested events into Dynatrace through API and Token.
If you have configured custom devices under Infrastructure observability > technologies and processes > custom devices.
The alerting profiles upon which the problem notification is configured.
The management zone rules configurations to include such event upon which the alerting profile is built.
If this entity doesn't have any tags, create a automatically applied tags based on the custom device attributes.
If this entity exists as a custom device without any belonging to a management zone, create a management zone to include it properly.

KR,

Peter.

Re: "My Event" Problem Alerts

StrangerThing — Fri, 11 Oct 2024 16:05:30 GMT

I'd start by querying events in DQL.

fetch events
| filter event.name == "My Event"

Then you'll get some more context around the event and problem that might help you determine where it came from.

Re: "My Event" Problem Alerts

ASE — Fri, 11 Oct 2024 18:09:47 GMT

Thank you for the suggestion, @StrangerThing.

All the Problem Alerts were event.provider = EVENTS_REST_API_INGEST, which does help our internal support team identify where they are coming from.

Re: "My Event" Problem Alerts

ASE — Fri, 11 Oct 2024 18:13:46 GMT

Thank you for the reply, @Peter_Youssef.

It was #1. There was some unexpected custom work being done outside of OneAgent and via the API.

Mystery solved. 😆

Re: "My Event" Problem Alerts

StrangerThing — Fri, 11 Oct 2024 19:11:08 GMT

Happy to help! I've had my fair share of "where the hell did this come from" in my Dynatrace adventure 🙂

Re: "My Event" Problem Alerts

Peter_Youssef — Sat, 12 Oct 2024 13:24:14 GMT

Good to know.

all the best @ASE .