topic Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm in Alerting

inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Aboud — Mon, 25 Nov 2024 08:46:46 GMT

I received the "Kubernetes PVC: Low Disk Space %" alarm, but I'm unsure why it was triggered. The threshold is set to 15%, and in reality, there is no disk space usage. and after I lowered the threshold to 14%, the issue was resolved — despite no disk space being used.

Could someone please help clarify why this is happening?

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Peter_Youssef — Wed, 13 Nov 2024 10:58:04 GMT

Hello @Aboud

It depends on the current Dynatrace version setup and as per the current configurations will find:

Dynatrace alert if the disk space is below "x" the assigned percentage.
For at least 3 minutes for example which means that the event should stay active for 3 minutes
Within the last 5 minutes as an example: means the events have been active as per the allocated percentage for at least 3 minutes during the last 5 minutes.
So as long as Davis AI observes the actual behavior of the monitored entities, it will trigger problem alerts upon fulfilling the matching criteria.
The same happens for the problem closure.

Hoping it adds value,

BR,

Peter.

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

p_devulapalli — Wed, 13 Nov 2024 11:03:02 GMT

@Aboud - try looking at the dt.kubernetes.persistentvolumeclaim.available and dt.kubernetes.persistentvolumeclaim.capacity for this PV and see what's being reported and compare it with what you see on the node, you might find some clue. Below link has more details

https://docs.dynatrace.com/docs/shortlink/alerts-k8s#persistent-volume-claims

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Aboud — Wed, 20 Nov 2024 06:40:35 GMT

@Peter_Youssef @p_devulapalli Thanks for your help. However, the matching criteria are not fulfilled, as the cluster capacity is 2 TiB, and the used space is 0%. Therefore, I don't understand why the alarm is showing that the available disk space is 0B. I have opened a ticket with the vendor, but they are still investigating the issue.

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Peter_Youssef — Wed, 20 Nov 2024 06:54:07 GMT

Hi @Aboud

Thanks for your feedback, If possible can you share the configured thresholds for alerting.

You can also check the default K8s related anomaly detections.

Hoping it will be sorted soon.😊

BR,

Peter

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Aboud — Sat, 23 Nov 2024 15:05:31 GMT

Hello @Peter_Youssef

Here is the threshold for alerting.

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Aboud — Mon, 30 Dec 2024 11:44:21 GMT

Hello @Peter_Youssef

FYI after investigation with DT Support we found that it was just a fake alarm, and we canceled it 🙂

Re: inquiry regarding "Kubernetes PVC: Low Disk Space %" Alarm

Peter_Youssef — Mon, 30 Dec 2024 11:56:11 GMT

Thanks @Aboud for your End of Year (EOY) 🌻 feedback 😉 😁, all the best.