topic Possible or not: Rise alert if there is no request for last 12 hours. in Alerting

Possible or not: Rise alert if there is no request for last 12 hours.

Romanenkov_Alex — Tue, 09 Jun 2020 12:19:02 GMT

Hello Community!

We have some specific service, that requests some information for 5 minutes.

This work can be done in 12 hours. (Night)

We need alert if for last 12 hours there aren`t requests (*/update ).

We can set custom threshold for response time of these requests.

How we can set alert if for last 12 hours there are no such request.

This service produce only /update requests, so it is not loaded. We have no static traffic on these service.

Are there some instruments for my case out-of-the-box?

Thanks.

Regards,

Alexander

Re: Possible or not: Rise alert if there is no request for last 12 hours.

ChadTurner — Tue, 09 Jun 2020 12:30:28 GMT

Interesting use case - im not 100% sure if you can set it for that, You can however set a maintenance window to ignore alerts in the time frame where there is no traffic.

The other thing is to allow DAVIS UI to baseline the data, so if there is no traffic from 6-6, then that will become a norm, and after a week or so, you wont get alerts of low traffic.

Very interesting use case, maybe others will have a solution other then a reoccurring daily maintenance window and letting DAVIS Baseline the data

Re: Possible or not: Rise alert if there is no request for last 12 hours.

gilgi — Mon, 15 Jun 2020 05:43:00 GMT

custom alerts can only be used to examine an hour interval max. Davis will learn the pattern for still as an hourly pattern, hence if your system one night has 5 requests at 11 and then 2 and 4 and then on the other night it's on different hours, DAVIS will probably won't be the answer for that.

As you're looking at a 12 hour interval and I assume these are the same hours, I would go for a simple API call from a shell being run at (let's say) 05:59, summing up the number of requests for the past 12 hours and if needed creates an alert.

You would need the metrics API and the events one. Should be relatively simple.

Gil.