topic Failed Datebase alerts configuration in Alerting

Failed Datebase alerts configuration

Jacksparrroww — Tue, 02 Jan 2024 09:15:29 GMT

Greetings, our client, an insurance company, has a problem,

The problem is as follows: there are several applications, these applications can get errors when going to the database (failed database connection errors), but some of these errors are not database related but application related, we are trying to understand this by looking at the applications. However, since our friends working on the night shift cannot understand, they can wake up the wrong teams or direct the problem to the wrong teams!

Is there a possibility that the database-sourced problem coming from the application that receives this error is correct or not, or is there a possibility to make a comparison between other applications and the database and tell us whether it exists or not?

Re: Failed Datebase alerts configuration

Mohamed_Hamdy — Mon, 25 Dec 2023 18:05:57 GMT

Hi @Jacksparrroww,

Is it possible to share the error details, you can hide any confidential data, as the issue on the database could be caused by the application or it could be a DB issue. and you can check the backtrace from the failed DB requests. also, you can create an alert according to the title of your inquiry or create a multidimensional analysis if needed.

Best Regards,

Mohamed

Re: Failed Datebase alerts configuration

Jacksparrroww — Tue, 26 Dec 2023 10:03:46 GMT

Hi Mohamed,

I've attached screenshots, how do you think we should proceed?

Re: Failed Datebase alerts configuration

Mohamed_Hamdy — Tue, 26 Dec 2023 16:43:24 GMT

Hi @Jacksparrroww

Thank you for sharing these screenshots, I can see the main issue, failed to connect to DB due to "pooled connection request timed out" and based on the screenshots, I'm not able to see that there is a spike in traffic to causing this issue from application side, so you need to check with DB team.

Also, you can get the details to get the stack trace and see a sample of the distributed traces and the backtrace to get more details.

as for the "pooled connection request timed out" please find more details here and here

Re: Failed Datebase alerts configuration

Jacksparrroww — Tue, 02 Jan 2024 08:35:12 GMT

I understand that you are stating that the problem is on the DB side, is that correct?

Also, can we interpret which side the error is on by looking at the increase in traffic in the same type of problems?

Re: Failed Datebase alerts configuration

Mohamed_Hamdy — Tue, 02 Jan 2024 17:04:04 GMT

you need to check for the traffic spike, you can check the traffic on the DB during this time from "AHEPRD" DB, you can share with us a screenshot from the AHEPRD traffic during that time and of course, you can notice if there is a spike in traffic or not during this time from the service page or requests graph, also, if the Service load spikes setting is enabled for services and DB services, you should detect problems for high load.

Note that Service load spikes and drops are disabled by default.